Bugs How to Report Bugs Effectively l l

Bugs

How to Report Bugs Effectively l l l Goal Bug life cycle Steps for submitting a bug Other observations Problems with Issue Tracking Systems

Goal of a bug report l l You submit a bug report to get it resolved “Enable the programmer to see the program failing in front of them” l l Show them in person Give detailed instructions on how to make it fail l l Programmers can gather additional info If they can make it fail, they will try to gather extra information until they know the cause. If they can't make it fail, they will have to ask you to gather that information for them.

Bug Life Cycle http: //en. wikipedia. org/wiki/Bugzilla#Design See “Steps for Submitting a bug” Developers read/add comments (see “Problems with Issue Tracking Systems”)

Steps for Submitting a Bug l First step is …

Steps for Submitting a Bug 1. Always search for a previous bug l l l Chances are already submitted the bug Add more information to the existing one (if necessary), rather than submitting a new bug https: //bugzilla. mozilla. org/enter_bug. cgi? product=Firefox&format=guided

Search for previous bugs to avoid submitting duplicates l http: //sourceforge. net/tracker/? func=detail&aid=917231&group_id=588&atid=100588

Steps for Submitting a Bug l Second step is …

Steps for Submitting a Bug 2. Provide enough information to allow the programmer to reproduce the bug l Exact steps to reproduce the bug l l l Which buttons Order Input (attach sample input file) Expected output Actual output Screenshot

Steps for Submitting a Bug 2. Provide enough information to allow the programmer to reproduce the bug l l l Software version, OS version Exact error message Stack trace Core dumps Other information (e. g. , “unexpected delays”)

Often, programmer will say “works for me”

Causes for “Works for Me”? l Cause #1 …

Causes for “Works for Me”? l Not enough information was provided by user l l l Some crucial information was omitted Submitter (not familiar with the features) thinks it is a problem, when it’s actually not Submitter is using an older software version with the bug, which has been fixed in the most recent version

Causes for “Works for Me”? l l Bug is stochastic, and due to its nondeterministic behavior it cannot be reproduced by developer Miscommunication due to unclarity of bug report l l Reporter meant something Developer understood something else, and could not reproduce bug

Steps for Submitting a Bug 3. Clarify the statements, verify the information, and submit l l Clear statements are essential Be specific l l "I selected Load" might mean "I clicked on Load" or "I pressed Alt-L” Be verbose l More information is better than less

Steps for Submitting a Bug 3. Clarify the statements, verify the information, and submit l Be careful of pronouns l l l Substitute “it” with the actual meaning. "I started Foo. App. It put up a warning window. I tried to close it and it crashed. " "I started Foo. App, which put up a warning window. I tried to close the warning window, and Foo. App crashed. ”

Steps for Submitting a Bug 3. Clarify the statements, verify the information, and submit l Read what you wrote l l Read the report and make it clear If there is a list of steps to reproduce the failure, follow them and see if you missed a step

Other Observations l When you go to a doctor, do you say “Doctor, I need a prescription for …”?

Other Observations l When you go to a doctor, do you say “Doctor, I need a prescription for …”? l l State your symptoms and doctor will suggest the appropriate remedy In a bug report describe the symptoms, not solutions l Enhance bug report with possible solution, but never omit the symptoms

Other Observations l Distinguish facts ("I was at the computer and this happened") from speculations ("I think the problem might be this") l Leave out speculations if you want to, but don't leave out facts

Other Observations l Not all bugs contain specific steps to reproduce: l l Fail every week Fail “randomly”

Example of “Random Occurring Bug” https: //bugzilla. mozilla. org/show_bug. cgi? id=412950

Problems with Issue Tracking Systems l Cannot delete or modify comments l l “Too much information” No threading l Discussion might be hard to follow

https: //bugzilla. mozilla. org/show_bug. cgi? id=412950 Issues?

https: //bugzilla. mozilla. org/show_bug. cgi? id=412950 Issues? “Conflicting” information Hard to follow discussions

Issues?

Issues? Hundreds of comments!

Similar bug submission process, different standards l l l Mozilla Google Chrome Java Oracle (Sun)

Example of Google Chrome Bug http: //code. google. com/p/chromium/issues/detail? id=70527

Example of Google Chrome Bug Additional info “Quality reviews”

Quality Review

Quality Review

Oracle (Sun) Bug Submission FAQ http: //developers. sun. com/resources/bugs. FAQ. html