User Studies Motivation January 30 2007 Usable Privacy
User Studies Motivation January 30, 2007 Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 1
How do we know whether security is usable? 2
Need to observe users n We are not our users! n (you may be surprised by what users really do) Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 3
Wireless privacy study n Many users unaware that communications over wireless computer networks are not private n How can we raise awareness? B. Kowitz and L. Cranor. Peripheral Privacy Notifications for Wireless Networks. In Proceedings of the 2005 Workshop on Privacy in the Electronic Society, 7 November 2005, Alexandria, VA. Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 4
Wall of sheep 5
Defcon 2001 Photo credit: Kyoorius @ techfreakz. org http: //www. techfreakz. org/defcon 10/? slide=38 6
Defcon 2004 Photo credit: http: //www. timekiller. org/gallery/Defcon. XII/photo 0003 7
Peripheral display n Help users form more accurate expectations of privacy n Without making the problem worse Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 8
9
Experimental trial n Eleven subjects in student workspace n Data collected by survey and traffic analysis n Did they refine their expectations of privacy? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 10
Results n No change in behavior n Peripheral display raised privacy awareness in student workspace n But they didn’t really get it Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 11
Privacy awareness increased “I feel like my information /activity / privacy are not being protected …. seems like someone can monitor or get my information from my computer, or even publish them. ” Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 12
But only while the display was on “Now that words [projected on the wall] are gone, I'll go back to the same. ” Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 13
Security and privacy indicators Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 14
Evaluating indicators n Case study: Privacy Bird Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 15
Platform for Privacy Preferences (P 3 P) n 2002 W 3 C Recommendation n XML format for Web privacy policies n Protocol enables clients to locate and fetch policies from servers Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 16
Privacy Bird n P 3 P user agent n Free download http: //privacybird. org/ n Compares user preferences with P 3 P policies Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 17
18
19
Critique Privacy Bird n Security people • Can attackers spoof it? • What if P 3 P policy contains lies? • Can P 3 P policies be digitally signed? • What about main-in-the -middle attacks? n Usability people • Green/red color blind problem • Do people notice it in corner of browser? • Do people understand privacy implications? • Why a bird? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 20
Typical security evaluation 21
Does it behave correctly when not under attack? n No false positives or false negatives Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 22
Anti-phishing tools Y. Zhange, S. Egelman, L. Cranor, and J. Hong. Phinding Phish: Evaluating Anti-Phishing Tools. In Proceedings of NSSS 2006, forthcoming. Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 23
Does it behave correctly when under attack? n Can attackers cause wrong indicator to appear? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 24
Correct indicator Wrong indicator Attacker redirects through CDN Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 25
Can it be spoofed or obscured? n Can attacker provide indicator users will rely on instead of real indicator? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 26
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 27
Usability evaluation 28
C-HIP Model n Communication. Human Information Processing (C-HIP) Model • Wogalter, M. 2006. Communication-Human Information Processing (C-HIP) Model. In Wogalter, M. , ed. , Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51 -61. Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 29
Do users notice it? n If users don’t notice indicator all bets are off n “What lock icon? ” • Few users notice lock icon in browser chrome, https, etc. n C-HIP model: Attention switch, attention maintenance Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 30
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 31
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 32
Do users know what it means? Web browser lock icon: “I think that it means secured, it symbolizes some kind of security, somehow. ” Web browser security pop-up: “Yeah, like the certificate has expired. I don’t actually know what that means. ” n C-HIP Model: Comprehension/Memory J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12 -14 July 2006, Pittsburgh, PA. Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 33
Cookie flag Netscape SSL icons IE 6 cookie flag Firefox SSL icon Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 34
Privacy Bird icons Privacy policy matches user’s privacy preferences Privacy policy does not match user’s privacy preferences Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 35
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 36
Do users know what to do when they see it? n C-HIP Model: Comprehension/Memory Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 37
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 38
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 39
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 40
Do users believe the indicator? “Oh yeah, I have [seen warnings], but funny thing is I get them when I visit my [school] websites, so I get told that this may not be secure or something, but it’s my school website so I feel pretty good about it. ” n C-HIP Model: Attitudes/Beliefs Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 41
Are users motivated to take action? n May view risk as minimal n May find recommended action too inconvenient or difficult n C-HIP Model: Motivation Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 42
Do they actually do it? “I would probably experience some brief, vague sense of unease and close the box and go about my business. ” n C-HIP Model: Behavior Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 43
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 44
Do they keep doing it? n Difficult to measure in laboratory setting n Need to collect data on users in natural environment over extended period of time n C-HIP Model: Behavior Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 45
How does it interact with other indicators? n Indicator overload? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 46
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 47
Summary: Security evaluation n Does indicator behave correctly when not under attack? • No false positives or false negatives n Does indicator behave correctly when under attack? • Can attackers cause wrong indicator to appear? n Can indicator be spoofed or obscured? • Can attacker provide indicator users will rely on instead of real indicator? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 48
Summary: Usability evaluation n Do users notice it? n Do they know what it means? n Do they know what they are supposed to do when they see it? n Do they believe it? n Are they motivated to do it? n Will they actually do it? n Will they keep doing it? n How does it interact with other indicators? Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong • http: //cups. cmu. edu/courses/ups-sp 07/ 49
- Slides: 49