ALGORITHMIC TRANSPARENCY QUANTITATIVE INFLUENCE Yair Zick National University

ALGORITHMIC TRANSPARENCY & QUANTITATIVE INFLUENCE Yair Zick National University of Singapore Joint work with Amit Datta, Anupam Datta, Ariel D. Procaccia and Shayak Sen

Black-Box Decision Makers are Everywhere Health Insurance discounts for data-driven insurance policies Educatio n Finance Media course assignments Determining loan eligibility Personalize d Ads Streaming and course allocation Credit Scores News & Social Media

Explaining Black-Box Decision Makers Stakeholders have a right to explanation … but algorithms are often “black boxes”: • intellectual property • inherently complex

Quantitative Feature Influence (Datta, Proccacia and Zick, IJCAI 2015; Datta, Sen and Zick, IEEE S&P 2016) ■ Numerical measures of feature importance in black-box settings: algorithmic transparency

Privacy Algorithmic Transparen cy Integrit y Fairnes s 5

Personalized Transparency Report An individual was deemed worthy of arrest Birth Year 1984 Drug History None Smoking History None Census Region West Race Black Gender Male ■ Evidence of racial discrimination ■ Illustrates the dangers of the black-box use of machine learning

Axiomatic Feature Influence (Datta, Procaccia and Zick, IJCAI 2015) ■ How should an influence measure behave? ■ What properties do we want our measure to have? ■ A common approach in game theory: provably fair revenue division methods. ■ Influence as a “resource” that needs to be divided. There exist unique influence measures satisfying certain natural properties.

Quantitative Input Influence (QII) (Datta, Sen and Zick, IEEE S&P 2016) Deals with correlated inputs • a causal measure • Breaks correlations via randomized interventions. Supports a general class of transparency queries • parametric in a quantity of interest • From individual to group influence Computes joint and marginal influence • uses game theoretic aggregation measures

QII for Individual Outcomes Inputs Classifier Quantity Measured

Quantity of Interest Outcome of an individual • Outcomes of a group of individuals • Disparity between group outcomes •

QII: Definition The Quantitative Input Influence (QII) of an input is the difference in a quantity of interest, when the input is replaced with a random value via an intervention.

Joint and Marginal Influence Classifier Interventions on individual inputs commonly have no effect! Decisio n

Set QII Inputs Classifier Quantity Measured

• • • The only value to satisfy certain desirable properties!

Personalized Transparency Report An individual was deemed worthy of arrest Birth Year 1984 Drug History None Smoking History None Census Region West Race Black Gender Male ■ Evidence of racial discrimination ■ Illustrates the dangers of the black-box use of machine learning

Summary General transparency queries Correlated inputs Parametric in a quantity of interest Define a causal measure Robust to broad range of queries Average Marginal Influence Additional Results Use gametheoretic notions of average marginal influence Most measures can be efficiently approximated Axiomatically justified measures QII measures can be made differentially private cheaply