Han Van Der Aa

Conclusion

This chapter concludes this doctoral thesis. Section 8.1 provides a summary of the main results. Section 8.2 discusses the implications of the presented work for research and practice. Finally, Section 8.3 provides an outlook on directions for future research.

8.1 Summary of Results

In this thesis, we focused on the automated analysis of process information contained in various representation formats. As a basis for the development of these techniques, we considered why different representation formats are used in organizations and the problems that can result from this situation. The main contributions of this thesis are provided by five techniques that focus on the alignment and comparison of process information in different informational artifacts. We can summarize the main results presented in this thesis as follows:

• An overview of the causes and consequences of process-information fragmentation. Based on an analysis of existing literature and a case study, we identified various reasons for the existence of a variety of process-information artifacts in organizations, as presented in Chapter 2. Most importantly, different artifacts are used in order to provide stakeholders with the process information that is relevant to conduct their tasks, in suitable representation formats. We identified three main consequences of this spread of process information, i.e., of process-information fragmentation. First, fragmentation can lead to increased time and effort required to maintain process information. Because organizations often struggle with these required efforts, this can also lead process information becoming outdated. Second, the fragmentation of process information can negatively affect the execution efficiency of processes. Ultimately, this leads to more time, effort, and costs required for their execution. Third, when fragmentation causes users to struggle to find the process information needed to perform their tasks, it can even lead to business process noncompliance. Such cases can lead to reduced quality of process outcomes and the incurrence of financial penalties.
• Inconsistency detection between process models and textual process descriptions. The use of textual process descriptions alongside process models makes process information more accessible to a variety of stakeholders. However, this comes with a clear risk that model and text become misaligned, for instance when process changes are not applied to both descriptions consistently. For organizations with hundreds of processes, the effort required to manually identify and clear up such conflicts is considerable. The approach presented in Chapter 3 addresses this problem by automatically detecting inconsistencies between a process model and a corresponding textual description. A quantitative evaluation using real-world data demonstrates that our approach achieves promising results. Particularly, our approach detected all model-text pairs in which a process model activity was not described in the text or where the order between process activities differed, with a high precision.
• Behavioral spaces as a means to capture behavioral uncertainty in processes. Uncertainty about conveyed process behavior plays an important role when analyzing semi-structured representation formats, such as textual process descriptions. Furthermore, uncertainty can result in the context of alignments between information from different artifacts. Given such uncertainty, it is often impossible to determine exactly which process behavior is conveyed by a process specification. Rather, there are numerous potential interpretations of the described process behavior. In order to still be able to reason about properties such as conformance in the context of uncertain behavior, this thesis introduced and applied the concept of a behavioral space as a means to capture all potential interpretation of behavioral uncertainty. We showed the need for and usefulness of behavioral spaces to capture uncertainty caused by alignments between process models and event logs in Chapter 4. Furthermore, Chapter 5 showed their application for the analysis of ambiguous textual process descriptions.
• Conformance checking in the context of behavioral uncertainty. By capturing behavioral uncertainty in processes using behavioral spaces, trustworthy conformance-checking results can be obtained in situations where existing conformance-checking techniques fail to do so. In this thesis, we showed the usefulness of behavioral spaces by using them in the context of conformance checking. The conformance-checking results obtained by our methods are probabilistic, differentiating between conforming, nonconforming, and potentially conforming behavior. We applied our method to two scenarios in order to illustrate the utility of behavioral spaces for conformance checking. Chapter 4 showed that behavioral spaces allow us to obtain conformance-checking results for numerous cases where traditional conformance-checking techniques cannot be applied. Similarly, Chapter 5 showed that the use of behavioral spaces provides a much better basis for conformance checking than other, naive ways of dealing with ambiguity in textual process descriptions.
• Transformation of natural language descriptions into measurable Process Performance Indicators (PPIs). The unstructured natural language used by managers to specify relevant PPIs differs considerably from the structured, formal definitions required to actually monitor them. The time and effort required to transform the natural language descriptions into a suitable format impedes the effectiveness with which organizations can monitor their process performance. To overcome this problem, Chapter 6 presented an approach that automates the transformation task. A quantitative evaluation with a set of real-world PPI descriptions and accompanying process models revealed that our approach generates PPI definitions that are highly similar to those created manually. Therefore, our approach represents a viable, automated alternative to an otherwise highly laborious and time-intensive, manual task. This enables organizations to more efficiently monitor the performance of their business processes and continuously adapt their monitoring activities to changing business needs.
• Process model matching based on event-log information. Process model matching provides the basis for many process analysis techniques, such as inconsistency detection and process querying. Chapter 7 introduced a technique that uses event-log information in process model matching. Our matching technique builds on six conceptual notions of similarity between event classes. Furthermore, we provided various operationalizations that quantify these conceptual notions. We demonstrated the usefulness of these similarity metrics through a quantitative evaluation using real-world data. The evaluation showed that by just considering the information specific to event logs, the introduced matchers can identify a considerable number of correspondences between event classes. By combining our instance-based matching techniques with traditional, model-based techniques users can strive to obtain matching results that cannot be obtained by using either of these techniques alone. In this way, the accuracy and, therefore, usefulness of techniques that build on process model alignments is improved.

To achieve these results, we followed internationally established standards applicable to IS research, conducted according to the behavioral science and design science paradigms [122]. Most importantly, this involved addressing relevant problems and evaluating our proposed solutions using appropriate metrics and relevant, real-world data collections. In this way, our results represent a significant contribution to the body of knowledge of the IS research discipline.

8.2 Implications

In this section, we consider the implications of the work presented in this thesis. Section 8.2.1 reflects on the implications for practice and Section 8.2.2 on the implications for research.

8.2.1 Implications for Practice

The work presented in this thesis has several implications for organizations that strive to improve their organizational efficiency and process quality. In particular, we identify the following main implications for practice:

• Insights for improved management of process information. Our analysis of the causes and consequences of process-information fragmentation, presented in Chapter 2, shows the importance of the proper management of informational artifacts for organizations. In particular, organizations should strive to make the right process information available to stakeholders in suitable formats. However, organizations should simultaneously ensure that this fragmentation does not result in a situation in which process cannot be maintained or found. A main threat in this regard is the ad-hoc creation of process-information artifacts, which reduces the traceability that exists between process information in different artifacts. Therefore, organizations should aim to establish guidelines for the creation and maintenance of process information. In this way, process information becomes easy to retrieve and stays in sync with the correct way to execute the process it describes.
• More efficient resolution of inconsistencies among process representations. Although organizations often use multiple process representation formats in order to provide different stakeholders with information in a suitable format, this comes with the clear risk that the different representations do not convey the same information about a process. Our inconsistency-detection technique presented in Chapter 3 enables organizations to automatically detect such inconsistencies between process models and textual process descriptions. By using our approach, organizations can more efficiently detect and, therefore, resolve inconsistencies that exist in their process documentation. In this way, organizations can save considerable amounts of time and effort. This especially applies to organizations with large process repositories, possibly containing information on hundreds of processes.
• Increased adaptability of process performance monitoring. The monitoring of business processes represents an important prerequisite for organizations to improve their processes and, in this way, maintain a competitive advantage. This monitoring task is greatly affected by the continuous changes to which business processes and their environments are subject. As a result, organizations should continuously adapt the way in which they monitor their processes to these shifts. Our transformation approach presented in Chapter 6 supports organizations in this endeavor. The approach bridges the Business-IT gap that exists between the managers that specify the performance indicators to be monitored and the systems engineers involved in the actual monitoring of these indicators. In particular, it enables managers to specify automatically measurable PPIs without precarious and time-consuming communication with system engineers. In this way, our approach enables organizations to more easily adapt the monitoring of their business processes to emergent changes.
• Improved detection and prevention of business process noncompliance. Business process noncompliance can pose considerable threats to organizations. Noncompliant actions can lead to reduced efficiency, reduced process quality, and even to the incurrence of financial penalties. In this thesis, we presented two novel conformance-checking techniques that improve the ability of organizations to detect nonconforming behavior. Chapter 4 presented a technique that enables the detection of nonconformance in a more trustworthy manner than traditional techniques. In particular, our technique avoids the need to impose, possibly incorrect, assumptions on the relations between observed events and a process model. Chapter 5 expands the applicability of conformance-checking techniques from merely structured process representations, to semi-structured representations, specifically in the form of textual process descriptions. In this way, we enabled conformance checking to take the wealth of process information that is stored in natural language documents into account. Ultimately, the presented techniques improve the detection of business process noncompliance, leading to increased organizational efficiency and process quality.

8.2.2 Implications for research

The work presented in this thesis has several implications for research in the field of business process management as well as IS research, in particular for the management and alignment of process information.

• Comparison and alignment of process information in semi-structured formats. Establishing alignments between process-information artifacts is an important prerequisite for the development of a variety of process analysis techniques, such as inconsistency detection, update propagation, and process querying. Most existing techniques focus on the creation of alignments between structured process representation formats, in the form of process models or process models and event logs. In this thesis, Chapters 3 and 6 presented techniques that align process information from other representation formats, specifically from textual process descriptions and natural language PPI descriptions. In this way, our research provides a basis for the development of analysis techniques that build on the alignments established by our techniques. Furthermore, our research also shows the potential for the development of further automated analysis techniques, specifically those that focus on process information contained in other semi-structured informational artifacts that have so far been largely ignored by research.
• Enabled reasoning in the presence of behavioral uncertainty. Uncertainty about process behavior plays a prominent role when analyzing unstructured or semi-structured process-information artifacts, as well as when reasoning based on automatically established alignments between artifacts. Our work presented in Chapters 4 and 5 has important implications in this regard. The introduction of the concept of behavioral spaces provides a structured manner to capture all possible interpretations of uncertain process behavior. By doing so, process analysis techniques can be applied without the need to select a single, possibly incorrect, interpretation of the process behavior. This demonstrates that the use of behavioral spaces opens up numerous possibilities for the development of process analysis techniques that were previously deemed unfruitful because of ambiguity or other causes of uncertainty.
• Improved process model matching through the consideration of event-log information. Alignments between process models provide an important basis for a variety of application scenarios and techniques. These alignments are, for instance, used to detect differences between models, for the harmonization of process model variants, process querying, and for the automated propagation of process changes. The accuracy and, therefore, usefulness of such techniques is highly dependent on the correctness and completeness of the alignments that are established by process model matching techniques. In Chapter 7, we showed that the quality of these results can be improved through the consideration of event-log information. In particular, by considering the contents of event logs in the matching tasks, alignment results can be obtained that model-based matchers cannot achieve. Therefore, our research supports the usefulness of analysis techniques that build on process model alignments because it can lead to an improved alignment quality. Furthermore, our results show that the consideration of instance-level information can be worthwhile in future development of process model matching techniques.

8.3 Future Research

This doctoral thesis focused on the fragmentation of process information and, in particular, on technological developments that can mitigate its consequences. The work presented in this thesis opens up possibilities for further research that pursues the efficient use and maintenance of process information within organizations. In this pursuit, we can distinguish two general perspectives to mitigate the negative aspects of process-information fragmentation: the technical and the organizational perspective. The technical perspective focuses on mitigation by providing technical solutions to specific problems caused by fragmentation. By contrast, the organizational perspective focuses on mitigation by reducing the extent of fragmentation itself.

8.3.1 Technical Perspective

This thesis primarily takes a technical perspective to mitigate the effects of process-information fragmentation. We presented conceptual and technical developments that focus on the more efficient use and maintenance of process information spread out over various artifacts. Different opportunities for future research stem from these developments.

A variety of scenarios in which organizations struggle with the impact of process-information fragmentation remain unaddressed by existing research. In particular, future research can focus on the development of techniques that (i) focus on more process perspectives, (ii) cover additional representation formats and (iii) that address additional use cases.

(i) Most of the techniques presented in this thesis focus on the comparison of process information from a control-flow perspective. While this is arguably the most relevant perspective in process analysis, it is important to recognize that the presented techniques could be extended to also consider, for example, the temporal, resource, and data perspectives. For instance, the consistency-checking technique from Chapter 3 could be extended to also detect inconsistencies with respect to the resources that should execute activities according to the process descriptions;
(ii) Regarding the coverage of additional representation formats, we primarily see opportunities to develop alignment and comparison techniques that focus on process information contained in slide sets, checklists, and spreadsheets. The use of these representation formats has been recognized in literature and observed in our case study, as discussed in Section 2.1.4. However, these formats are currently not yet covered by existing alignment techniques;
(iii) Aside from broadening the spectrum of considered formats, techniques can also be developed that address use cases beyond the comparison and alignment of process information. For instance, our technique presented in Chapter 3 aims to detect inconsistencies between process model and textual descriptions, which can be used to detect changes between the process representations. Instead, by automatically propagating the effects of process changes from one representation to the other, process consistency can be ensured, rather than just detected. The conceptual contributions presented in this thesis can be particularly helpful in the pursuit of such new research directions. For instance, the approach used to extract process information from textual process descriptions and textual PPI descriptions can be utilized to support the extraction of process information from other semi-structured artifacts. Furthermore, the concept of a behavioral space can be applied to use cases in which the extraction or alignment of process information cannot provide deterministic results.

In this thesis, and in the above reflection, we have so far focused on application scenarios of alignment techniques in isolation. Typically, existing research focuses on the alignment or comparison of two informational artifacts. However, as discussed in Section 2.1.4, realistic settings can involve much larger numbers of informational artifacts related to a single process. It is for those processes that the problems caused by fragmentation can be the most pressing. To better support such scenarios, research should strive for integrated solutions that enable the use and maintenance of all information related to a process and, even, entire process repositories, rather than consider pairs of informational artifacts in isolation. The main prerequisite for such support is that traceability should be established between process information from all artifacts. Note that the provision of holistic support through such techniques differs considerably from the use of data warehouses to accomplish similar goals. Data warehouses require a considerable overhaul of the way in which organizational members capture process information in the first place. By contrast, by using techniques such as presented and proposed in this thesis, organizational members can still document process information in the format that they prefer, whether this is a process model, a textual description, or any other representation format of choice.

8.3.2 Organizational Perspective

Although this thesis mainly focused on the technological perspective, the organizational perspective also plays an important role when dealing with process information in organizations. In particular, by taking this perspective into consideration, the negative aspects associated with process-information fragmentation can already be partially avoided, reducing the need for their later mitigation. This thesis, in particular Section 2.1.4, already showed the crucial nature of taking such steps to reduce the effects of fragmentation. This can, for instance, be achieved by introducing guidelines that change the way in which organizations capture process information. However, further research is required in order to better understand how to achieve the desired effects. Currently, research that addresses the question of how to best capture process information has primarily focused on the suitability of representation formats for specific tasks (cf. [55, 93]). A main problem is that these scenarios and, therefore, also the usefulness of representation formats are typically considered in isolation, generally using experimental settings. There is limited reflection on how or why these representation formats co-exist in practical settings, let alone that these questions are investigated. Nevertheless, insights into this direction are necessary, because it is this co-existence that leads to problems when using and maintaining process information. A main direction to investigate in this regard is the trade-off that exists between the benefits of using more artifacts to provide stakeholders with the information they need versus the negative aspects associated with such further process-information fragmentation. Put differently, when does the creation of more process-information artifacts help stakeholders by providing them with the information they need and when does it reduce the ability of stakeholders to actually find this information? By answering such questions, important insights can be obtained that will enable organizations to manage their process information in a more considerate manner. Therefore, such insights would help organizations to further improve their efficiency and provide higher quality results.