What is Explainable AI Mark Elliot National Centre

What is Explainable AI? Mark Elliot National Centre for Research Methods 1

What’s the problem? ● Legal requirements – Transparency ● – a data controller “must be able to demonstrate that personal data are processed in a transparent manner in relation to the data subject. ” (GDPR Art 5. 2) Accountability ● A “data controller shall be responsible for, and be able to demonstrate compliance with” the principles of GDPR (art 5. 2)

What’s the problem? ● Legal requirements – Right to explanation ● Right to access “meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject” (GDPR Article 13 – and elsewhere)

What’s the problem? ● Human system requirements ● Many AI system’s are black boxes

Drivers for explanation ● Why did the system pick this rather than that? ● When will it fail? ● Is the system trustworthy? ● Locus of control.

Vanilla Machine Learning system Initial parameter values Algorithm Recommendation Decision Prediction ML process Training data Data

Types of why question ● ● ● Why did the algorithm work that way? What elements of the problem space led to that decision? Why did your learning process produce that algorithm?

Types of why question ● Internal ● External

Options ● ● Post hoc: produce an separate algorithm which reads the end to end process In built: Build the decision making algorithm so that traces have within them the basis for explanation.

Personalisati on ● One size does not fit all ● Complexity of explanation – ● ● Assumptions of knowledge/expertise Assumption about type of explanation required Implies models of the user – which is a whole extra problem….

Unintended consequences of explanability ● We are adding an extra constraint into a system designs – so in terms of the task the results may be non-optimal.

Unintended consequences of explanability ● We are constraining the range of algorithms and learning processes to ones that humans can understand. – So we are channelling AI into the human replication