Cybersecurity and artificial intelligence The European Commissions science

Cybersecurity and artificial intelligence The European Commission’s science and knowledge service Joint Research Centre

The European Commission’s science and knowledge service Joint Research Centre Cybersecurity and artificial intelligence Emerging trends Dr. Ignacio Sanchez 2

The Joint Research Centre (JRC) at a glance JRC's Mission "As the science and knowledge service of the Commission our mission is to support EU policies with independent evidence throughout the whole policy cycle" 3000 staff Almost 75% are scientists and researchers. Headquarters in Brussels and research facilities located in 5 Member States. 3

Cyber & Digital Citizens’ Security Unit Mission To strengthen trust and security of the European Citizen in a sustainable and inclusive ICT-based European society by scientific research on how emerging Information and Communication Technologies will impact on the security and privacy of citizens’ daily life. To work on risk mitigation, on cybersecurity, cybercrime, data protection, privacy and on the associated legal and regulatory frameworks aiming at a balance between European security needs and fundamental citizen rights including from the perspective of the emerging Digital Single Market. 4

European Commission's Joint Research Centre (JRC) flagship report on Artificial Intelligence Joint Research Centre’s report on Artificial Intelligence (5 December 2018) ISBN: 978 -92 -79 -97217 -1 (online), 978 -92 -79 -97219 -5 (print), 978 -92 -79 -98213 -2 ISSN: 1831 -9424 (online), 1018 -5593 (print), 1831 -9424 DOI: 10. 2760/11251 (online); 10. 2760/936974 (print); 10. 2760/91283 URL: https: //ec. europa. eu/jrc/en/publication/eur-scientific-and-technicalresearch-reports/artificial-intelligence-european-perspective Coordinated Plan on Artificial Intelligence (7 December 2018) Communication from the Commission to the European Parliament, the European Council, the European Economic and Social Committee and the Committee of the Regions - Coordinated Plan on Artificial Intelligence (COM(2018) 795 final) Communication Artificial Intelligence for Europe (25 April 2018) Communication from the Commission to the European Parliament, the European Council, the European Economic and Social Committee and the Committee of the Regions on Artificial Intelligence for Europe. 5

Artificial Intelligence “Artificial intelligence (AI) refers to systems designed by humans that, given a complex goal, act in the physical or digital world by perceiving their environment, interpreting the collected structured or unstructured data, reasoning on the knowledge derived from this data and deciding the best action(s) to take (according to pre-defined parameters) to achieve the given goal. AI systems can also be designed to learn to adapt their behaviour by analysing how the environment is affected by their previous actions. As a scientific discipline, AI includes several approaches and techniques, such as machine learning (of which deep learning and reinforcement learning are specific examples), machine reasoning (which includes planning, scheduling, knowledge representation and reasoning, search, and optimization), and robotics (which includes control, perception, sensors and actuators, as well as the integration of all other techniques into cyber-physical systems). ” European Commission's High-level expert group on Artificial Intelligence. "A definition of AI: main capabilities and scientific disciplines", 18 December 2018 6

Machine learning A new paradigm What has changed? More data More computing power New algorithms AI When using machine learning, human developers no longer program an algorithm to tell the computer how to solve a given problem but instead they program it to teach the computer to learn how to solve the problem. 7

Cybersecurity and AI & DT: 4 angles of influence 1 Resilience 4 2 Deterrence 3 Opportunities 1. AI to create smarter cybersecurity 2. Robustness/Vulnerability of AI algorithms E. g. more effective security controls (antivirus, intrusion detection and prevention, email scanning), automatic discovery of vulnerabilities, automatic malware analysis, smarter cyber defence, etc. E. g. adversarial machine learning (i. e. exploitation of weaknesses in AI algorithms to change their behavior), attacks against AI powered cyberphysical systems, etc. 4. Use of AI to fight cyber attackers & criminals 3. Misuse of AI E. g. better biometrics, smarter forensics, fraud analysis, encryption challenge, fight against fake news, etc. E. g. creation of deep audio video fakes, AI powered malware, smarter social engineering attacks, etc. Conceptual model depicting the logical links between the different components of the cybersecurity risk in the context of the influence of AI and Digital Transformation 8 Challenges

AI to create smarter cybersecurity ANGLE 1 | Smarter Cybersecurity Smarter cybersecurity controls Automatization in labour intensive tasks • Better biometric controls • Automatic discovery of vulnerabilities • Network Intrusion Detection and Prevention systems • Automatic exploitation of vulnerabilities AI applied to malware analysis • Malware detection • Security logs and event correlation • Email filters (spam, phishing, etc. ) • • Automatization of security operations (e. g. incident response) • Cybersecurity exercises (attack/defense) and training • Awareness raising 9

Autonomous security operations ANGLE 1 Example | Smarter Cybersecurity 2016 Cyber Grand Challenge Final Event DARPA Cyber Grand Challenge (CGC) URL: https: //www. darpa. mil/program/cyber-grand-challenge 10 Real time visualisation of a buffer overflow exploit in CGC

Machine learning: a new paradigm ANGLE 2 | Robustness/Vulnerability of AI algorithms • AI cannot be made unconditionally safe, like any other technology. • AI software based on machine learning techniques is found to not be robust against malicious attacks that exploit a specific set of AI vulnerabilities. • Thus: AI itself will likely be exploited and misused, for example by criminals. • Of highest importance: securing cyberphysical systems employing AI. 11

Adversarial Machine Learning ANGLE 2 Example | Robustness/Vulnerability of AI algorithms Example of adversarial Machine Learning in image recognition Classified as: dog. Add crafted noise. Classified as: ostrich. Fooling a state-of-the-art object recognition neural network through the simple act of Introducing specially crafted noise (Szegedy et al. 2013) 12

Adversarial Machine Learning ANGLE 2 Example| Robustness/Vulnerability of AI algorithms Left: real graffiti on STOP sign. Right: designed stickers to mimic the situation and fool an object recognition AI. >= 83 % success rate in fooling a state-of-the-art object recognition system in real drive-by tests “Robust Physical World Attacks on Deep Learning Models” (Eykholt et al. 2018) Example of a real life adversarial attacks 13

Influence of AI in cybersecurity threats ANGLE 3 | Misuse of AI New type of cyberattacks Automatization of labour • Usage of AI for the evasion of cybersecurity controls (e. g. captcha solvers, antispam/antiphising filters, malware detection) • Some cybercrime labour intensive tasks can be automated, enabling for example: • Massive spear phishing attacks (highly targeted) • Autonomous / automatic decision making in cyberattacks (e. g. autonomous lateral movements in cyberattacks) • Highly interactive and scalable social engineering attacks (e. g. chat bots over IM, email or social media) • Automatic botnet management • Enhanced social engineering and identity theft (i. e. realistic imitation of drafting style, voices and video) • Faster, more interactive and scalable attacks • Automatization of cybercrime processes (e. g. ransomware payments) • Abuse of AI limitations More interactive More effective AI can make cyberattacks … 14 More targeted Scalable Cheaper More difficult to attribute Faster More autonomous

AI-enhanced cyberattacks ANGLE 3 Examples | Misuse of AI AI-based attacks against authentication systems Captchas Social Engineering, deception and Identity theft and social engineering • Deep video fakes • Fake audio voice • Fake news Phishing and cyber extortion https: //github. com/mwrlabs/captcha_cracking AI applied to malware • Autonomous decision making in malware (lower dependence on remote control) • Evasion of malware detection controls • Self-maintained, self-protecting and evolving botnets Brundage, Miles, et al. , "The malicious use of artificial intelligence: forecasting, prevention, and mitigation”, ar. Xiv preprint ar. Xiv: 1802. 07228 (2018) Sivakorn, Suphannee, Jason Polakis, and Angelos D. Keromytis, "I’m not a human: Breaking the Google re. CAPTCHA”, Black Hat (2016) 15 • More targeted (e. g. impersonation of friends; location of targets in social media) • More contextualized (e. g. language translation, use of personal information) Automated interactive attacks • Massive spear phishing • Automatic ransom negotiation and assistance

AI in Law Enforcement ANGLE 4 | AI to fight (cyber)crime AI can help in: New AI based tools: • Reinforcing investigative capabilities • Strengthen digital evidence-making in court • • WHAT Biometrics Image / audio / video analytics Localisation fingerprinting Evidence identification and correlation Encryption challenge Automatic scanning Digital forensics (data in transit, data at rest) Analysis of local and global context WHERE WHO AI will help not only in the fight against cybercrime but also cyber-dependant crime and more generally crime 16

Automatic Tattoo recognition ANGLE 4 Example | AI to fight (cyber)crime From classical text-based search (keywords) … Visual Tattoo Inspection Keyword(s) 1 st candidate Keyword DB 2 nd 3 rd 1 st candidate 2 nd 3 rd Matches HUMAN/SKULL, OBJECT/FIRE, OBJECT/WEAPON • Human image interpretation is subjective • The process is tedious and therefore error-prone • The precision of recognition depends on the tattoo description To JRC ML / Deep Learning feature-based approach … Deep Learning Model Tattoo Detection Segmentation Deep Learning Model Real datasets used: FBI Tattoo Database (NIST/Tatt-C) JRC's research on Automatic Tattoo recognition 17 Learnt features Tattoo Identification Deep Learning or SVM Tattoo Description

Smarter image /audio/video forensics ANGLE 4 Example | Camera identification SPN extraction Camera fingerprinting From picture to video JRC's research on smarter image / audio / video forensics 18

Stay in touch • EU Science Hub: ec. europa. eu/jrc • Twitter: @EU_Science. Hub • Facebook: EU Science Hub - Joint Research Centre • Linked. In: Joint Research Centre • You. Tube: EU Science Hub 19