ADBIS 2007 Varna Bulgaria 03 10 2007 Towards
ADBIS 2007; Varna, Bulgaria; 03. 10. 2007 Towards Self-Optimization of Message Transformation Processes Matthias Böhm 1, 2, 3 *, Dirk Habich 2, Uwe Wloka 3, Jürgen Bittner 1, and Wolfgang Lehner 2 SQL Gmb. H Dresden, Germany 2 Dresden University of Technology, Database Technology Group 3 University of Applied Sciences Dresden, Database Group 1
Outline • Introduction • Integration Platform Trans. Connect® • Process Optimization Techniques • Summary and Conclusion 2 / 28
Outline • Introduction • Integration Platform Trans. Connect® • Process Optimization Techniques • Summary and Conclusion 3 / 28
Introduction • starting point – integration of heterogenous information systems – horizontal service integration by message-based communication using the Message Transformation Model (MTM) • motivation / problem description – suboptimal modeled processes – dynamic workload characteristics – total costs of ownership • contribution towards self-optimization – first rule-based optimization techniques – first workload-based optimization techniques – prototypical implementation within Trans. Connect 4 / 28
Introduction • Message Transformation Model (MTM) 5 / 28
Introduction • Message Tansformation Model (MTM) – Message Model Hierarchical message structure – Process Model (reconsidered) • Interaction-oriented activ. • Control-flow-oriented activ. • Data-flow-oriented activ. Base model "Directed Graph" 6 / 28
Introduction • Message Tansformation Model (MTM) – Example Process 7 / 28
Outline • Introduction • Integration Platform Trans. Connect® • Process Optimization Techniques • Summary and Conclusion 8 / 28
Integration Platform Trans. Connect • Trans. Connect – message based application integration – inbound adapters – outbound adapters – process engine • Trans. Connect 1. 3. 6 overall architecture 9 / 28
Integration Platform Trans. Connect • Trans. Connect 1. 3. 6 Server architecture 10 / 28
Integration Platform Trans. Connect • Component Process. Parser 11 / 28
Integration Platform Trans. Connect • External Layer: WSBPEL 2. 0 process <process xmlns=""> <!--declarations--> <!– process description--> </process> 12 / 28
Integration Platform Trans. Connect • Conceptual Layer: MTM process type 13 / 28
Integration Platform Trans. Connect • Internal Layer: JAVA process plan public class es_process 1 extends Process. Plan { private Internal. Message msg 1 = null; private Internal. Message msg 2 = null; @Override protected Internal. Message execute. Node(Internal. Message input) throws MTMException { try { Invoke node 1 = new Invoke("sap_mq", "DEQUEUE", AService. OTYPE_RECEIVE); node 1. set. IDs(get. PTID(), get. PID(), get. NID()); msg 1 = node 1. execute( msg 3 ); } catch( MTMSignal. Exception mse ) { /*signal handling*/ } /*. . . */ } } 14 / 28
Integration Platform Trans. Connect • Component System. Monitor – interval monitoring / continuous monitoring – determination of suboptimal process plans – recompilation of process plans adaptive optimization strategies – Self-Optimization according to IBM MAPE concept (Monitor, Analyse, Plan, Execute) 15 / 28
Integration Platform Trans. Connect • Component System. Monitor – Inbound monitor events: performance measurement -- average process type execution time -- (not normalized!) SELECT AVG(End. Time - Start. Time) FROM Processing. Performance WHERE NID = -1 AND -- node type process PID IN ( SELECT PID FROM Process WHERE PTID = (SELECT PTID FROM Process. Type WHERE Name=‘es_process 1‘)) 16 / 28
Outline • Introduction • Integration Platform Trans. Connect® • Process Optimization Techniques • Summary and Conclusion 17 / 28
Process Optimization Techniques • influencing-factors – optimization aim: throughput / execution time – execution knowledge: statistics / ad-hoc – optimization techniques: rule-based / workload-based • technique classification – Rule-based process optimization • Control flow optimization • Data flow optimization – Workload-based process optimization • Message indexing • Control flow optimization • Data flow optimization 18 / 28
Process Optimization Techniques • Rule-based process optimization – Control flow optimization • Redundant control flow elimination • Unreachable subgraph elimination • Preventing local subprocess invocation 19 / 28
Process Optimization Techniques • Rule-based process optimization – Data flow optimization • • • Double Variable Assignments Unnecessary Variable Declarations Two sibling Tanslation operators Unnecessary Switch-paths Two sibling validations Basically these techniques are adopted from imperative programming language compilers 20 / 28
Process Optimization Techniques • Workload-based process optimization – Message indexing 21 / 28
Process Optimization Techniques • Workload-based process optimization – Message indexing – Control flow optimization • query scrambling techniques (external systems delay, network delay elimination, and disk I/O delay) • parallel flow management 22 / 28
Process Optimization Techniques • Workload-based process optimization – Data flow optimization • Switch operator optimization 23 / 28
Process Optimization Techniques • Evaluation Experiment - "Complex Integration Process" rule-based and workload-based process plan rewriting 24 / 28
Process Optimization Techniques • Evaluation Experiment - "Complex Integration Process" – average inbound message size: 7 KB 25 / 28
Outline • Introduction • Integration Platform Trans. Connect® • Process Optimization Techniques • Summary and Conclusion 26 / 28
Summary and Conclusion • Summary – optimization techniques were illustrated – implementation and evaluation prove the high optimization potential – lots of further research items along • Conclusion – research of optimization techniques will be displaced from the grounding systems to the integration process • Future work – DIPBench (Data-Intensive Integration Process Benchmark) – GCIP (Model-Driven Generation and Optimization of Complex Integration Processes) – MIX (Message Indexing for Document-Oriented Integration Processes) – Adaptive Enterprise Integration Platform 27 / 28
ADBIS 2007; Varna, Bulgaria; 03. 10. 2007 Towards Self-Optimization of Message Transformation Processes Matthias Böhm 1, 2, 3 *, Dirk Habich 2, Uwe Wloka 3, Jürgen Bittner 1, and Wolfgang Lehner 2 SQL Gmb. H Dresden, Germany 2 Dresden University of Technology, Database Technology Group 3 University of Applied Sciences Dresden, Database Group 1
- Slides: 28
