BIG DATA EUROPE Platform concept and architecture Application
BIG DATA EUROPE Platform concept and architecture Application to data-intensive science Yiannis Mouchakis Institute of Informatics and Telecommunications NCSR “Demokritos” gmouchakis@iit. demokritos. gr 23 January 2017 NCSR
Big Data Europe (CSA: 201517) Show societal value of Big Data o Lower barrier for using big data technologies o o Across all societal challenges addressed by Horizon 2020 Effort and resources to convert tools and workflows Skills and expertise Help establish data value chains o Across languages, organizations, and domains www. big-data-europe. eu 23 -févr. -21
Consortium NCSR DEMOKRITOS Semantic Web and linked data expertise SW/LD technologies key to our we make it easy for you pitch
Architecture Big Data Integrator (BDI): o The prototype developed by BDE Main points of the architecture o o o Dockerization Support layer, including integrated UI Semantification layer www. big-data-europe. eu 23 -févr. -21
Platform Architecture App Layer Support Layer Init Daemon GUIs Traffic Forecast Platform Layer Base Setup Real-time Stream Monitoring Satellite Image Analysis Spark Flink Kafka . . . Resource Management Layer (Swarm) Semantic Layer Ontario SANSA Hardware Layer Cloud (AWS, GCE, Azure, …) Semagrow Data Layer Hadoop Premises . . . RDF NOSQL Stores Elastic. Search Cassandra . . .
BDE Docker Containers www. big-data-europe. eu 23 -févr. -21
Pilots www. big-data-europe. eu 23 -févr. -21
BDI use cases demonstrated Flexibility to existing workflows o Scaling out o o Drug discovery platform ported to new data storage infrastructure Sensor data analysis software trivially distributed to multiple streams Image analysis and traffic pattern algorithms in Spark and Flink Preprocessing and ingestion o o Distributed data architectures pre-process and maintain big data Not needed simultaneously or reduced to small data www. big-data-europe. eu 23 -févr. -21
Thank you for your attention! Questions? www. big-data-europe. eu Big. Data. Europe action Web site: https: //www. big-data-europe. eu Big Data Integrator: https: //github. com/big-data-europe Semagrow: http: //semagrow. github. io SANSA: http: //sansa-stack. net 23 -févr. -21
- Slides: 9