Catalog of Formalized Application Integration Patterns

07/06/2018 ∙ by Daniel Ritter, et al. ∙ 0

Enterprise application integration (EAI) solutions are the centrepiece of current enterprise IT architectures (e.g., cloud and mobile computing, business networks), however, require the formalization of their building blocks, represented by integration patterns, verification and optimization. This work serves as an instructive pattern formalization catalog that leads to the formalization of all currently known integration patterns. Therefore, we explain the classification of the underlying requirements of the pattern semantics and formalize representative patterns from the different categories, by realizing them in timed db-net. In this way, the catalog will allow for the addition of future patterns by assigning them to a category and applying the described formalism.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The enterprise integration patterns (EIPs) from 2004 [3] denote messaging patterns that serve as building blocks, when implementing an enterprise application integration (EAI) system [5]. While the EIPs are still practically relevant today [12, 8], the emerging technological, social and business trends since then require pattern extensions, e. g., for integration adapters and endpoints [7], exception handling and fault tolerance [10, 11], as well as information security among many other aspects [9, 8]. With the totality of those patterns playing a major role in real-world application integration architectures, the lack of a comprehensive formalization of the single pattern semantics and their compositions (beyond the currently only attempt using plain coloured petri nets (CPN) [2, 1]) will be instrumental for the verification and optimization of the current and future EAI process modeling and architectural solutions [8].

While we found a suitable formal representation in the recent work on db-nets [6] and extended them to timed db-nets for the formalization of EIPs, this work strives to collect and formalize all patterns from the aforementioned, currently known pattern catalogs, by being a catalog of formalized patterns itself. However, with a total amount of patterns and approximately that are due to formalization (i. e., no meta concepts), a complete catalog of formalized patterns would not be practical and lead to many repetitive formalizations of the same underlying concepts.

Consequently, we categorized the underlying requirements for the patterns’ semantics (i. e., data and control flow, external resources, transactions, complex message formats, and time) and discuss representative patterns from each of the categories (e. g., data-control, data-time patterns) and their realizations in timed db-net in Sect. II:

  • Data, transact. resource

  • Control, transacted resource, time

  • Control-time

  • Data, transact. Resource Time

With these representatives of each category, the others can be easily formalized due to the same underlying semantic concepts, making this work rather an instructive catalog manual, than a simple pattern reference. The new patterns identified after denote a case on how to add and formalize potentially new patterns beyond this work, since they strengthened existing and added new categories.

Then in Sect. III, we briefly discuss how the timed db-net formalism helps to experimentally test the correctness of the patterns–again by category and not for each pattern– and Sect. IV concludes this work.

Ii Formalized Patterns by Category

In this section, we define selected patterns from the requirement categories discussed before. The subsequent categories have been chosen, since the go beyond the already existing formalisms and in their combination they allow for a complete representation of the known as well as future patterns.

Ii-a Data, transact. resource patterns: Resequencer

The stateful Resequencer is a pattern that guarantees the order of messages in (asynchronous) communication [3]. Figure 1 shows the resequencer in timed db-net representation.

Fig. 1: Resquencer pattern

The entering message msg contains sequence (seq) and order (ord) information and is persisted in the database, represented by a db-net view place . For the first message of a sequence the sequence will be created in view place Message Sequences, and for all subsequent message of that same sequence, the messages are stored. As soon as the sequence is complete, i. e., all messages of that sequence arrived, the messages of that sequence are queried from the database in order by the Reorder transition. Eventually, the messages are forwarded in ascending order to .

Ii-B Control, transacted resource, time patterns: Circuit Breaker

The Circuit Breaker pattern [8] addresses failing or hang up remote communication, which impacts the control flow of a Request-Reply pattern [3] by using transacted access to external resources. Figure 2 shows a request-reply representation in timed db-net, extended by a circuit breaker ?wrapper? (using db-net view places) that protect the remote call.

Fig. 2: Circuit breaker

At the beginning, the circuit is closed, i. e., communication is enabled. In case of an exception (exc) during send or a timeout of 30 time units during receive, the number of failed attempts (nexc) is increased and stored into the Enpoints view place, which maintains a list of all endpoints (epid) and their consecutive failures. If the number of failures reaches a limit (e. g., nexc ) the circuit trips, and thus updates the status entry in view place Circuits to ?open?, which let all subsequent messages immediately go to the exception place (exc). The circuit breaker can be closed by manually updating the status to ?closed? to give the remote endpoint another chance. With additional logic, self-resetting mechanisms can be implemented.

Ii-C Control-time patterns: Throttler, Delayer

The following patterns mostly require control flow and time aspects, and are thus in timed CPNs with guards.

The Throttler pattern helps to ensure that a specific receiver does not get overloaded by regulating the number of transferred messages. Figure 3(a) shows the realization of a throttler that emits five messages per second to the receiving place .

(a) Throttler
(b) Delayer
Fig. 3: Control-time patterns

A slightly different pattern of this category is the delayer, as shown in Fig. 3(b), which uses a timer to reduce the frequency of messages sent to the receiving place .

Ii-D Data, transact. Resource Time patterns: Aggregator

The combination of data, transacted resources and time aspects in patterns makes them the semantically most complex ones. For example, Fig. 4 specifies the semantics of a commonly used stateful Aggregator [3] pattern.The aggregator persistently collects messages in a special timed db-net view place and aggregates them in an Aggregate PN transition based on a completion condition (e. g., sequence isComplete) or on timeout, depending on a sequence seq, represented as PN guards. For this an incoming message msg is correlated (cf. correlate) to an existing sequence based on its content. If the message is the first of a sequence, a new sequence and a message to sequence assignment is created in a persistent store called Message Sequences. If a message correlates to an existing sequence, which is aggregated due to a timeout isExpired, the update fails. Then a roll-back is executed (red reverse timed db-net arc) that puts the message back to the message channel (PN Place). Now, this message matches first msg and a new sequence is created accordingly.

Fig. 4: Aggregator pattern.

Iii Correctness Testing of timed db-net Patterns

The correctness of an integration pattern realization represented in timed db-net can be validated by evaluating the execution trace on the persistence layer. According to the timed db-net execution semantics, a pattern produces several B-snapshots during the execution of the pattern from an input snapshot with database instance to a final snapshot , denoted by ,..,. In case of a database instance is not compliant with , then the execution stops and leaves the timed db-net in an intermediate state , otherwise is the final state. Hence, the control flow can be validated, by checking, whether the pattern produces a token to the correct final database instance. Figure 5 gives a schematic view of an timed db-net pattern, for which the inner workings are unknown and the data is exchanged through input places , output places , and an intermediate place in that subsumes all exceptional places, together with the corresponding database intances , , and . The input or newly created tokens eventually manifest in entries in the DB instances.

Data and (transacted) resouce-bound patterns. With respect to data, format , transacted resources and exceptional situations, for a given instance with test data , either an expected final persistent database instance with the correct schema or an expected error state must be produced by the pattern. Otherwise the pattern is incorrect with respect to its definition for the requirements.

Patterns with msg. channel order. Similarly, the channel execution order can be validated. In case of the content-based router, an initial instance will result a different output instance , depending on the values in

and the routing conditions. While this can be checked as for the first case, the balancer requires a sequence of input instances, which then have to produce data entries in the output instances that fit the probability values and distribution of the balancer (e. g., Kolmogorov-Smirnov test 

[4]).

Fig. 5: Throttler testing (schematic)

Time-bound patterns. Finally, a timed pattern can be validated by assigning timestamps time to the database instances (as ?on-insert timestamps? in actual databases). This allows for checking delays by comparing the insert timestamps , of data to instance and those of . With this, a numeric delay can be checked by and a message per time ratio by counting the number of entries #(e) inserted to within time buckets , making pattern valid, if expected.

For instance, Fig. 5 shows a throttler pattern, provided with test messages into its input place in , bound to a database instance in . The transition takes the messages from one after the other (i. e., capacity guard cap=1) and inserts them into with instance , where the messages are picked up by a timed transition Timer and moved to , and thus slowing down the processing. The data logic layer mediates by rewriting the database instances from the input , to the output . Thereby the average size of denotes the size of the time buckets .

Example: flawed pattern implementation.

In the absence of a tool that allows for the verification (model-checking) of a pattern, we test their correctness through simulation, by example of a ?flawed? content-based router implementation. Therefore, we use our timed db-nets CPN Tools extension111Demonstrator available for download, 10/2018: https://github.com/dritter-hd/db-net-eip-patterns; containing a flawed implementation of a content-based router, and the non-flawed implementation of the aggregator pattern from Fig. 4.

A content-based router, is a pattern that takes one input message and passes it, read-only to exactly one receiver. This is done by evaluating a condition per recipient on the content of the message. Figure 6 shows one out of many router implementations, which look correct, however, violates this definition on the data and not the control level.

Fig. 6: Flawed content-based router: transition T enabled

For the evaluation we use the aforementioned method for ?data and (transacted) resource-bound patterns?, which is based on the reachability of a correct database state. Such a correct state would be a database instance with data in table channel1 and an empty channel2 table.

Now, let us explore the inner workings of this implementation using timed db-net. In Fig. 6, transition T reads the token in place I and then conditionally inserts it to the two subsequent places.

Fig. 7: Flawed content-based router: messages duplicated according to the conditions / transition guards

Since the value of the token matches all conditions, both output places and receive a copy of the token, shown in Fig. 7. In terms of application integration, this could mean that two companies receive a payment request or a sales order that was actually meant for only one of them. In the net, the two subsequent transitions and are enabled and fire by executing the database inserts , while being the respective database table of the receiver. From the net alone, the semantics seem to be correct. However, on the persistence layer, no correct state has been reached. This is illustrated by looking into the database instance after the tokens have been processed successfully on the control layer (cf. Fig. 8(a), Fig. 8(b)).

(a) channel1
(b) channel2
Fig. 8: Database instance after content-based router PN was executed ?successfully?

In the persistence layer, both tables are filled with data, which is an invalid state according to the definition of the content-based router. Hence, the deep insight into the process and corresponding data aspects of timed db-net allow for detecting flaws in the pattern implementations as well as richer information for fixing it.

Iv Conclusion

In this work, we define and discuss the formalization of important integration patterns per category, and thus contribute an instructive catalog of pattern realizations and a description of testing their correctness for the current and future patterns to come. The power of the chosen timed db-net formalism becomes especially clear for complex patterns, since for the first time, all of their underlying data, transactional resource and time semantics become explicit now.

References

  • [1] D. Fahland and C. Gierds. Using petri nets for modeling enterprise integration patterns. Technical Report BPM Center Report BPM-12-18, 2012.
  • [2] D. Fahland and C. Gierds. Analyzing and completing middleware designs for enterprise integration using coloured petri nets. In CAiSE, pages 400–416, 2013.
  • [3] G. Hohpe and B. Woolf. Enterprise integration patterns: Designing, building, and deploying messaging solutions. Addison-Wesley, 2004.
  • [4] A. N. Kolmogorov. Sulla determinazione empirica di una leggi di distribuzione. Giorn. 1st it lit о Ital. Attuari, 4, 83, 91, 1933.
  • [5] D. S. Linthicum. Enterprise Application Integration. Addison-Wesley, 2000.
  • [6] M. Montali and A. Rivkin. Db-nets: On the marriage of colored petri nets and relational databases. T. Petri Nets and Other Models of Concurrency, 12:91–118, 2017.
  • [7] D. Ritter and M. Holzleitner. Integration adapter modeling. In CAiSE, pages 468–482, 2015.
  • [8] D. Ritter, N. May, and S. Rinderle-Ma. Patterns for emerging application integration scenarios: A survey. Information Systems, 67:36 – 57, 2017.
  • [9] D. Ritter and S. Rinderle-Ma. Toward A collection of cloud integration patterns. CoRR, abs/1511.09250, 2015.
  • [10] D. Ritter and J. Sosulski. Modeling exception flows in integration systems. In EDOC, pages 12–21. IEEE, 2014.
  • [11] D. Ritter and J. Sosulski. Exception handling in message-based integration systems and modeling using BPMN. Int. J. Coop. Inf. Syst, 25(2), 2016.
  • [12] O. Zimmermann, C. Pautasso, G. Hohpe, and B. Woolf. A decade of enterprise integration patterns: A conversation with the authors. IEEE Software, 33(1):13–19, 2016.