Using Evidence Effectively CS 4001 Mary Jean Harrold

Using Evidence Effectively CS 4001 Mary Jean Harrold 1

Star Criteria S T A R CS 4001 Mary Jean Harrold 2

Star Criteria Sufficiency: Is there enough evidence? Typicality: Are the chosen data representative and typical? Accuracy: Are the data accurate and up to date? Relevance: Are the data relevant to the claim? CS 4001 Mary Jean Harrold 3

Kinds of Evidence 1. From personal experience a. Personal experience b. Personal observation or field research 2. From interviews, questionnaires, surveys 3. From reading and research 4. Statistical Hypothetical examples, cases, scenarios 5. Reasoned sequence of ideas CS 4001 Mary Jean Harrold 4

Does this evidence meet the STAR criteria? Claim: Testing a software system before deployment typically costs a company millions of dollars Evidence: It cost at least this much to test the Boeing 777 airplane before it was deployed What kind of evidence is this? CS 4001 Mary Jean Harrold 5

Does this evidence meet the STAR criteria? Claim: Software errors cost companies millions of dollars per year Evidence: June 2002 National Institute of Standards and Technology (NIST) report found that software bugs are so prevalent and so detrimental that they cost the U. S. economy an estimated $59. 5 billion annually or about 0. 6% of the GDP. What kind of evidence is this? CS 4001 Mary Jean Harrold 6

Does this evidence meet the STAR criteria? Claim: Test cases developed for large software systems typically cover only about 30% of the functionality of the software Evidence: In the development of Java Architecture for Bytecode Analysis (JABA) within my research group, the test suite we developed covers approximately 30% of the statements in the program. What kind of evidence is this? CS 4001 Mary Jean Harrold 7

Does this evidence meet the STAR criteria? Claim: One of the best ways to improve the security of your software is to use good software engineering practice and process Evidence: This was stated by Prof. Eugene Spafford, Purdue University in his Distinguished Lecture for Georgia Tech Information Security Center in 2001. What kind of evidence is this? CS 4001 Mary Jean Harrold 8

Does this evidence meet the STAR criteria? Claim: On-line discussion of ethical issues improves students’ willingness and ability to participate in discussions (over class discussions) Evidence: CS 4001 d on-line chat conducted by Jim Hudson What kind of evidence is this? CS 4001 Mary Jean Harrold 9

Perspective, Angle of Vision Need to understand writer’s perspective when using evidence to support claim Two pieces of evidence 1. One of the best ways to improve the security of your software is to use good software engineering practice and process (E. Spafford…) 2. You cannot have a secure system without having a secure operating system (Jonathan Spapiro, researcher for Extremely Reliable Operating System (EROS), in his GTISC talk, 2004) CS 4001 Mary Jean Harrold 10