SOA for Hierarchical GUI Test Case Generation Using

SOA for Hierarchical GUI Test Case Generation Using Automated Planning. by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. Presented by Bob Mazzi 9/30/08

GUIs – Older than you think • First GUIs – Xerox PARC 1981 • Windows GUIs 1985 – Present – 85 -94 Installed within DOS – 95 – Present True Standalone OS • Apple GUIs 1984 – Present • IBM OS/2 1987 – 2006 ( RIP )

State of the Art GUI = Graphic User Interfaces M. M Kaddah “ Interactive Scenarios for the development of a Unser Interface Prototype” 93 GUI Design B. A. Meyers “Why are Human. Computer Interfaces Difficult to design and Implement? ” 7/93 D. Rosenberg “User Interface Prototyping in the 90’s” 93 R. M. Mulligan, M. W. Altom, D. K. Simkin “User Interface Design…” 91 A. Kaster “User Interface Design…” 91 General Test Processes See Slide 4 GUI Tools B. A. Meyers “User Interface Software Tools” 95 GUI Testing A. Howe etal “Test Case Generation as an AI Planning Problem” 97 M. L. Hammontree etal “ Integrated Data capture and Analysis Tools for research and testing a GUI” 5/92 L. White “Regression Testing of GUI Event Interactions” 11/96 A. Walworth “Java GUI Testing” 98 M. G. El-Said, G. Fischer, S. A Gamalei-Din, M Zaki “ADDI A Tool for Automating the design of visual Interfaces” 97 L. The “Stress Tests for GUI Programs” 9/92 J. Nielson “Iterative User Interface Design” 11/93 L. R. Kepple “The Black Art of GUI Testing” 2/94 Planning of Testing See Slide 5 This Paper April 2000 D. J. Kasik, H. G. George “Toward Automatic Generation of Novice User test scripts” 4/96 Subsequent Work

State of the Art / General Testing T. S. Chow “Testing Software Design Modeled by Finite-State Machines” ” 3/78 S. Esmelioglu etal “Automated Test case Generation Planning and Reporting” 10/97 J. M. Clarke “Automated Test Generation from a Behavioral Model” 5/98 P. J. Bernhard “A Reduced Test Suite for Protocol Performance testing” 7/94 H. Cho etal “Redundancy Identification/Removal and Test Generation for Sequential circuits…” 7/93 R. K. Shehady etal“A method to Automate User Interface Testing …” 6/97 General Test Processes. to Slide 3

State of the Art / Test Planning M. Peot, D. Smith “Conditional Nonlinear Planning” 6/92 D. S. Weld “An introduction to Least Commitment Planning” 94 D. S. Weld “Recent Advances in AI Planning” 99 A. L. Blum, M. L. Furst “Fast Planning through Planning Graph Analysis” 97 K. Erol et al “HTN Planning: Complexity and Expressivity” 8/94 H. Kautz, B. Selman “ The Role of Domain Specific Knowledge in the planning as Satisfiability Framework” 98 J. Koehler etal “Extending Planning Graphs to the ADL Subset” 97 Test Planning Processes. To Slide 3

Overall Timeline • • • 1981 Early GUI Precursors ( Xerox / PARC ) 1983 First Apple GUI ( Lisa ) 1985 First Microsoft GUI ( Win 1. 0 ) 1987 First IBM GUI ( OS/2 ) 1978 -97 – General topic of “Testing” papers 1991 -93 – GUI Design papers 1992 -98 – GUI Testing papers 1995 -97 – GUI Tools papers 1999 – This paper

Related Work • Atif M. Memon – – – – • Martha E. Pollack – • 3 Common papers, no other related papers Qing Xie – • 3 Common papers, no other related papers Jaymie Strecker – • 3/4 Common papers, no other related papers Xun Yuan – • 7 Common papers, no other related papers Adithya Nagarajan & Ishan Banerjee – • 6 Common papers, no other related papers Mary Lou Soffa – • 6 Papers with M. Pollack and M. L. Soffa 1 Paper with M. L. Soffa 4 papers with Adithya Nagarajan 3 papers with Ishan Banerjee 3 Papers with Xun Yuan 3 Papers with Jaymie Strecker 3 papers with Qing Xie 2 papers with Scott Mc. Master 3 Common papers, One other related paper? Scott Mc. Master – 2 Common papers, no other related papers

Subsequent Related Work by Atif Memon • • Atif Memon's Publications 2009 “Using Reverse Engineering for Automated Usability Evaluation of GUI-Based Applications” by Atif M. Memon. “Testing Graphical User Interfaces” by Jaymie Strecker and Atif M. Memon. • • • 2008 “Using a Pilot Study to Derive a GUI Model for Automated Testing ” by Qing Xie and Atif M. Memon. “Test Case Generator for GUITAR” by Daniel Hackner and Atif M. Memon. “Relationships Between Test Suites, Faults, and Fault Detection in GUI Testing” by Jaymie Strecker and Atif M. Memon. “Call-Stack Coverage for GUI Test-Suite Reduction ” by Scott Mc. Master and Atif M. Memon. “Automatically Repairing Event Sequence-Based GUI Test Suites for Regression Testing” by Atif M. Memon. “Alternating GUI Test Generation and Execution” by Xun Yuan and Atif M. Memon. • • • 2007 “Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction” by Scott Mc. Master and Atif M. Memon. “Designing and comparing automated test oracles for GUI-based software applications” by Qing Xie and Atif M. Memon. “Covering Array Sampling of Input Event Sequences for Automated GUI Testing” by Xun Yuan, Myra Cohen. And Atif M. Memon, “Automated GUI Testing Guided by Usage Profiles” by Penelope Brooks and Atif M. Memon. “An event-flow model of GUI-based applications for testing” by Atif M. Memon. “Agile Quality Assurance Techniques for GUI-Based Applications” by Qing Xie and Atif M. Memon. “Faults' Context Matters” by Jaymie Strecker and Atif M. Memon. “Using GUI Run-Time State as Feedback to Generate Test Cases” by Xun Yuan and Atif M. Memon. • • • 2006 “Model-Based Testing of Community-Driven Open-Source GUI Applications” by Qing Xie and Atif M. Memon. “Employing user profiles to test a new version of a GUI component in its context of use” by Atif M. Memon. “Studying the Characteristics of a `Good' GUI Test Suite” by Qing Xie and Atif M. Memon. “Call Stack Coverage for GUI Test-Suite Reduction” by Scott Mc. Master and Atif M. Memon. “Techniques and processes for improving the quality and performance of open-source software” by Adam Porter, Cemal Yilmaz. • • • 2005 “Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software” by Atif M. Memon and Qing Xie. “Rapid "Crash Testing" for Continuously Evolving GUI-Based Software Applications” by Qing Xie and Atif M. Memon. “Call Stack Coverage for Test Suite Reduction” by Scott Mc. Master and Atif M. Memon. “Automating regression testing for evolving GUI software” by Atif Memon, Adithya Nagarajan, and Qing Xie.

Subsequent Related Work by Atif Memon • • • 2004 “Empirical Evaluation of the Fault-Detection Effectiveness of Smoke Regression Test Cases for GUI-Based Software” by Atif M. Memon and Qing Xie. “Automated GUI Regression Testing Using AI Planning” by Atif M. Memon. “A Distributed Continuous Quality Assurance Process to Manage Variability in Performance-intensive Software” by Arvind S. Krishna, Cemal Yilmaz. Atif M. Memon, Adam Porter, Douglas C. Schmidt, Aniruddha Gokhale, and Balachandran Natarajan, “Using Tasks to Automate Regression Testing of GUIs” by Atif M. Memon. • • • 2003 “Regression testing of GUIs” by Atif M. Memon and Mary Lou Soffa. “Advances in GUI Testing” by Atif M. Memon. “What Test Oracle Should I Use for Effective GUI Testing? ” by Atif M. Memon, Ishan Banerjee, and Adithya Nagarajan. “DART: A Framework for Regression Testing Nightly/daily Builds of GUI Applications” by Atif M. Memon, Ishan Banerjee, and Adithya Nagarajan. “ Automatically Testing “Nightly/daily Builds” of GUI Applications” by Atif M. Memon, Ishan Banerjee, and Adithya Nagarajan. • • 2002 “GUI Testing: Pitfalls and Process” by Atif M. Memon. • • • 2001 “Hierarchical GUI Test Case Generation Using Automated Planning” by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. ( This paper ) “Coverage criteria for GUI testing” by Atif M. Memon, Mary Lou Soffa, and Martha E. Pollack. • “A comprehensive framework for testing graphical user interfaces” by Atif M. Memon. Ph. D. dissertation, 2001. Advisors: Mary Lou Soffa and Martha Pollack; Committee members: Prof. Rajiv Gupta (University of Arizona), Prof. Adele E. Howe (Colorado State University), Prof. Lori Pollock (University of Delaware). • • 2000 “Automated test oracles for GUIs” by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. “A Planning-Based Approach to GUI Testing” by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. “Plan Generation for GUI Testing” by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. • • 1999 “Using a goal-driven approach to generate test cases for GUIs” by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. • •