DataParallel Transcoding for the 3 DInternet Masters Thesis

Data-Parallel Transcoding for the 3 D-Internet Master‘s Thesis Final Presentation Al-waleed Shihadeh, 10 th November, 2014, Garching Software Engineering for Business Information Systems (sebis) Department of Informatics Technische Universität München, Germany wwwmatthes. in. tum. de

Agenda § § § Business Vision Problems Thesis Contribution Implementations System Evaluation Outlook Al-waleed Shihadeh 2014 © sebis 2

Business Vision • Use the Digital Mockup (DMU) as key entry point and reference to support communication, collaboration and data exchange • Develop A web-based collaborative engineering environment that : - Integrates different engineering domains. - Allows engineers to have different views for the same 3 D model Al-waleed Shihadeh 2014 © sebis 3

Problems 3 D models are very large more than 1 GB § Extract shapes into separate files § Loading the 3 D model shapes in background. § Create web resources for each of the shapes in the 3 D model. Browsers issues § Transcode the extracted shapes to binary format. § Compress the size of the shapes. § Enhance the performance of the 3 D viewer. Problem § Transcoding large 3 D models takes very long time consuming task § Ariane 5, 84 MB , 879 shapes, more than 40 hours. Al-waleed Shihadeh 2014 © sebis 4

Thesis Contribution § Data-parallel transcoding approach for X 3 D data § Distribute the transcoding process. § Decrease transcoding time § Hadoop framework and Map. Reduce programming model § Evaluation and performance analysis of the proposed approach § Local evaluation : Airbus machines § Remote evaluation : Amazon Elastic Compute Cloud (EC 2) Al-waleed Shihadeh 2014 © sebis 5

Implementations Al-waleed Shihadeh 2014 © sebis 6

Phase 1 : Pre-parsing Objectives: § Remove unwanted nodes. § Replace USE nodes in the X 3 D file. § Create product breakdown tree. Al-waleed Shihadeh 2014 © sebis 7

Phase 2 : Partitioning Objectives § First Extract All Shapes: § Extract All parts. § Extract The root part. XQuery and Base. X Al-waleed Shihadeh 2014 © sebis 8

Phase 2 : Map. Reduce Tasks § Setup the environment § Install required libraries: Java, openssh, pydoop, etc. . § Install and setup Hadoop cluster. Cluster Type Nodes Memory (GB) OS CPU Single node cluster 1 8 Ubuntu- 64 bit 2. 93 GHz Dual node cluster 2 8 and 2 Ubuntu- 64 bit 2. 93 GHz, 2. 5 GHz Multiple node clusters 4 8 Ubuntu- 64 bit 8 v. CPUs, 2. 5 GHz § Design and implement Map. Reduce jobs § Using Custom JAR technology. § Using Hadoop streaming technology. Al-waleed Shihadeh 2014 © sebis 9

Phase 4 : Deployment At this stage… - Each of the phases is performed manually. - Map. Reduce job results : transcoded files & shapes web resources. There is a need for a web application that has the following features: - Allows end users to transcode their 3 D models automatically - Create and deploy web pages for viewing the transcoded models - Enable users to specify the transcoding commands - Provide basic functionality for the implemented 3 D viewer - 3 D interaction features such as rotating the 3 D model, zoom in and zoom out, hide and show the parts of the 3 D model, etc. Al-waleed Shihadeh 2014 © sebis 10

Phase 4 : Deployment Al-waleed Shihadeh 2014 © sebis 11

Evaluation § Local Evaluation - The goal is to select the approach with the best performance for further evaluation - Performed on the local machines at Air. Bus - Using the Hadoop single and dual node clusters § EC 2 Evaluation - Evaluate only the best approach Evaluate the effects of both environment and system parameters System parameters (Shapes Number , Files number, Split size, shapes type) Environment parameters (CPU power, Memory size, Node number) Al-waleed Shihadeh 2014 © sebis 12

Local Evaluation Al-waleed Shihadeh 2014 © sebis 13

EC 2 Evaluation: System Parameters Al-waleed Shihadeh 2014 © sebis 14

EC 2 Evaluation: Environment Parameters Al-waleed Shihadeh 2014 © sebis 15

Summary & Outlook • Local solution vs. EC 2 solution. § Local solution § § § Very expensive Requires log time to be prepared. EC 2 solution § § § Access to powerful machines. Easy and quick to construct. Security risk • Enhance the 3 D Viewer § Add more 3 D functionality like picking and selecting specify parts. § Enhance the loading of the 3 D model. • Move the 3 D model information to a database § Parts and shapes names and descriptions. § Engineers comments. Al-waleed Shihadeh 2014 © sebis 16

Questions? Al-waleed Shihadeh Technische Universität München Department of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße 3 85748 Garching bei München Tel Fax +49. 89. 289. 17100 +49. 89. 289. 17136 a. shihadeh@tum. de wwwmatthes. in. tum. de

Backup Slides Al-waleed Shihadeh 2014 © sebis 18

Extract Shapes Al-waleed Shihadeh 2014 © sebis 19

Transcode Geometries Al-waleed Shihadeh 2014 © sebis 20

Local Evaluation Al-waleed Shihadeh 2014 © sebis 21

HDFS Hadoop Custom JAR Al-waleed Shihadeh 2014 © sebis 22

Demo Al-waleed Shihadeh 2014 © sebis 23