Why Johnny Cant Encrypt A Usability Evaluation of
Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5. 0 Presented by Yin Shi
Overview u Introduction u Understanding the Problem u Cognitive Walkthrough u User Test u Conclusion
Introduction Effective security requires a different usability standard u Security mechanisms are effective only when used correctly u – Matt Bishop claimed that configuration errors are the cause of more than 90% of all computer security failures Making security usable will require the development of domain-specific user interface design principles and techniques u Choose PGP 5. 0 for our case study u – Designed by general consumer software standards – “Significantly improved graphical user interface makes complex mathematical cryptograph accessible for novice computer users. ”
Understanding the Problem u Defining Usability for Security – Definition: Security software is usable if the people who are expected to use it: u Are reliably made aware of the security tasks they need to perform u Are able to figure out how to successfully perform those tasks u Don’t make dangerous errors u Are sufficiently comfortable with the interface to continue using it
Understanding the Problem u Problematic Properties of Security – Five inherent properties of security u The u The u unmotivated user property abstraction property lack of feedback property barn door property weakest link Property A Usability Standard for PGP – Need for privacy and authentication – What needs to be done – How to do it and avoid dangerous errors
Evaluation Methods u Two Methods – An informal cognitive walkthrough – A user test performed in a laboratory
Cognitive Walkthrough u Visual Metaphors (keys) – PGP’s user interface relies on graphical depictions of keys and locks – Improvements u An extension of the metaphor to distinguish public keys for encryption and private keys for decryption u Different icons for public and private keys
Cognitive Walkthrough u Visual Metaphors (signatures) – The icon of the blue quill pen is used to indicate signing is problematic – Quill pen icon will not help user understand they need to use their private keys to generate signatures – Improvements u Keep quill pen to represent signing, but modify it to show a private key as the nib of the pen u Use some entirely different icon for signatures
Cognitive Walkthrough u Different Key Types – Originally, PGP used the RSA algorithm for encryption and signing – PGP 5. 0 uses the Diffie-Hellman/DSS algorithm – PGP 5. 0 can handle RSA keys, but other version PGP can’t handle DSS keys – Lack of forward compatibility u Recipients with RSA keys can’t decrypt it verify signatures – PGP 5. 0 alerts its users to this compatibility issues in two ways
Cognitive Walkthrough u Different Key Types – Uses different icons to depict the different key types – When user attempt to encrypt documents using mixed key types, a warning message is showed – Improvement u Double-clicking on a key pops up a Key properties window
Cognitive Walkthrough u u u Metaphor of choosing people Human icons obscure the key type information Better to display multiple keys that person owns
Cognitive Walkthrough u Key Server – Are publicly accessible databases – PGP offers three key server operations under the Keys pull-down menu
Cognitive Walkthrough u Problems with the presentation of the Key Server – Users may not realize that it exists u No representation of it in the top level of PGPkeys display – PGPkeys keeps no records of key server access – PGP’s key revocation operation does not send the resulting revocation certificate to the key server
Cognitive Walkthrough u Key Management Policy – Two ratings for each public key u Validity – how sure the user is that the key is safe to encrypt with u Trust – how much faith the user has in the key – May not realize PGP can automatically sets the validity rating of a key based on whether it has been signed by a certain number of sufficiently trusted keys.
Cognitive Walkthrough u Irreversible Actions u Consistency u Too Much Information – Accidentally deleting the private key – Accidentally publicizing a key – Accidentally revoking a key – Forgetting the passphrase – Failing to back up the key rings – encoding – PGPkeys application presents the user with too much information to make sense of u Owner’s name, validity, trust level, creation date, and size u Nothing to help the user figure out which parts of the display are the most important to pay attention to
User Test u Test Design u Initial task is to send the secret message to the team members in a signed and encrypted email u Main steps – Generate a key pair, get the public keys – Make their own public key available to team members – Type the secret message into an emails – Sign the email using private key, encrypt the email using the team member’s public keys
User Test u One of the member had an RSA key u Participant would encounter mixed key types warning message u Each of the five campaign members was represented by a dummy email account and a key pair: – These were accessible to the test monitor through a network laptop u The test monitor could send email to the participant from the appropriate dummy account
User Test (Results) u Avoiding dangerous errors – Three of them accidentally emailed the secret without encryption – One forgot her passphrase u Figuring out how to encrypt with any key – One couldn’t figure out how to encrypt at all – A reconfiguration of PGP may required – Another one kept sending unencrypted test messages, and finally succeeded after being prompted to use the PGP plug in buttons
User Test (Results) u Figuring with out the correct key to encrypt – 11 participants figured out how to encrypt, but failed to understand the public key model – Another one so completely misunderstood the model that he generated key pairs for each team member rather than for himself u Decrypting an email message – Five participants received encrypted email – One can’t figure how to decrypt it – Two took a very hard time to figure it out – Other two were able to decrypt without any problem
User Test (Results) u Publishing the public key – Ten could make their public key available to the team members – Two never addressed key distribution – Those ten, five sent their keys to key server – Three emailed to the team members – Other two did both u Getting other people’s public keys – Eight successfully got the team members’ public keys – The others either never seemed aware they need other people’s public key, or they did know how to get it
User Test (Results) u Handing the mixed key types problem – Only four managed to send encrypted email correctly – One didn’t have mixed key types problem – The other three received a reply email for complaining that they couldn’t decrypt email Signing an email message u Verifying a signature on an email message u Creating a backup revocation certificate u – Only three participants managed to successfully send encrypted email and decrypt a reply – In response to direct prompting for backup u One didn’t send the key pair to the key server u One sent email to the campaign manager u One simply ignored the prompt
User Test (Results) u Deciding whether to trust keys from the key server – Of the eight participants, only three expressed some concern over if they should trust the keys – None of the three made use of the validity and trust labeling provided by PGPKeys
Conclusion/Questions PGP 5. 0’s user interface does not come even reasonably close to achieving our usability standard u It does not make public key encryption of electronic mail manageable for average computer users u Public work on usability evaluation in a security context would be extremely valuable u We expect to find better design strategies u
- Slides: 23