Loading data with Talend UWs approach to loading

Loading data with Talend UW’s approach to loading data into Kuali Student August 23, 2012 9: 30 am – 11: 30 pm

Agenda § § Introductions Product Review Dave’s Job Design Demo

Introductions David Mc. Clellan is lead ETL developer for UW’s My. Plan and KS CM implementation. David is a Senior Java Developer with 25 years of programming experience.

Talend Overview Talend is: § An open source company § Provides middleware for data and application integration § Wide product suite including ETS, MDM, Data integration, Governance, ESB, BPM. Free/Premium versions UW applied Talend: § Extract, transform and integrate data from UW’s Student Database into Kuali Student CM database § Data includes ATP, Org, Course

Talend + - Uses Eclipse Documentation pretty good (for less complicated connectors) Job design is not well documented 100’s of Prebuilt connectors Only two WS connectors, one broken. Won’t generate attributes that we need Join across databases and different data sources Forum Support (ok) Quality is erratic: Eclipse Plugins are erratic, buggy Open Source Possible to write custom Java code Require Java Developer Part of a broader integrated product line Best features are only available in premium version

Job design - Process 1. Create properties file 2. Configure connector to source database (user/pass/query to run/jdbc url etc) 3. Design mapping between source and target destination (contains transformations) 4. Configure Web Service connector 5. Generate java classes needed by Web Service connector 6. Run job 7. Review log

Job Design - Desktop

Demo § ATP (simple) § Course (complex)