Profiling and Testing Applications Using the Eclipse Test

Profiling and Testing Applications Using the Eclipse Test and Performance Tools Platform (TPTP) Eugene Chan, IBM Tivoli Software Valentina Popescu, IBM Tivoli Software Paul Slauenwhite, IBM Rational Software November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 1

Agenda Ø p p p Overview of TPTP Profiling tools TPTP Testing tools What’s new? November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 2

Overview - Eclipse TPTP ØEclipse top level project • 2002 : Eclipse tools subproject - Hyades • 2004 : Eclipse top-level project - TPTP Ø Mission: • To build a generic, extensible, standards-based platform for test and performance tracing tools. Ø Goals: • Platform of choice for test, performance, and monitoring tools. • Exemplary tooling. • Enable value-added third-party tooling through extensibility and high-quality APIs. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 3

Overview - TPTP project structure Ø Comprised of four projects • Platform • Test • Trace • Monitoring Ø Developed by a number of strategic contributors • Active contributors: Intel, IBM, OC Systems • Inactive contributors: Scapa Technologies, Computer Associates, Compuware, FOKUS Ø For more information visit http: //www. eclipse. org/tptp/ November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 4

Overview - TPTP framework Ø TPTP offers a common, extendable framework as well as reference implementations for the following functions: • • • Testing Profiling Static code analysis Static and dynamic Bytecode Instrumentation (BCI) Application monitoring and log analysis November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 5

Overview - TPTP Test tools Ø Common framework for testing tools thereby integrating disparate test types and execution environments. • Common perspectives and views for interacting with target systems and resources. • Reference navigators, viewers, editors and wizards through extension points. • Standard EMF data model, query framework and assets repository. • Common data collection and execution framework on local and remote targets. Ø Reference implementations for the TPTP Test framework: • Manual testing. • JUnit and JUnit Plug-in testing. • URL testing. • Automated GUI Recording/Playback. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 6

Overview - TPTP Profile tools Ø Common framework for profiling simple Java applications or complex Web applications running on multiple platforms, on different hosts. • User interface actions to interact with the profiled application. • Start, stop, attach, detach from the profiled application. • Views framework used to analyze the profiled application. • Views can be extended and customized using extension points. Ø Reference implementation of the Profiling framework: • Java Profiling tools based on JVMPI and JVMTI. • Actions to interact with the profile application: run garbage collection, collect object references, filter and sort data • Views to analyze collected data: performance, memory analysis, thread views Ø Probekit, a generic tool for probe creation and insertion. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 7

Demo Ø Eclipse TPTP: Ø 4. 4. 0. 3 all-in-one Ø API Recorder Technology Preview Ø Demo Java Application: Ø Simple store November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 8

Agenda ü Ø p p Overview of TPTP Profiling tools TPTP Testing tools What’s new? November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 9

Profiling Tools Ø Profiling Perspective November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 10

Profiling Tools Ø Start a profiling session… November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 11

Profiling Tools Ø Profiling Filter November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 12

Profiling Tools Ø Profiling Option – Memory Analysis November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 13

Profiling Tools Ø Open Profiling View November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 14

Profiling Tools Ø Profiling View - Object Allocations view November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 15

Profiling Tools Ø Profiling View – Object Allocation View November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 16

Profiling Tools Ø Profiling – Object Allocation November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 17

Profiling Tools Ø Profiling Option – Execution Time Analysis November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 18

Profiling Tools Ø Profiling View – Execution Statistic View November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 19

Profiling Tools Ø Profiling View – Execution Statistics View November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 20

Profiling Tools Ø Profiling View – Open Source Action November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 21

Profiling Tools Ø Profiling – Execution Analysis November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 22

Profiling Tools Ø Profiling Option – Thread Analysis November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 23

Profiling Tool Ø Profiling View – Thread Analysis View November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 24

Profiling Tool Ø Profiling View – Call Stack November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 25

Profiling Tool Ø Profiling Views – Toolbar Actions November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 26

Profiling Tool Ø Profiling On Sever Ø Simplify web application profiling. Ø A pre-configured shortcut to launch and attach to web application for profile. Ø Ø Context menu Servers view November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 27

Profiling Tool Ø Help > Help Contents November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 28

Agenda ü ü Ø p Overview of TPTP Profiling tools TPTP Testing tools What’s new? November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 29

Testing Tools – Basic Concepts Ø Test Perspective Ø Set of navigators, viewers, editors and wizards for tests and test assets. Ø Test Suite Ø Consists of test cases and behaviors. Ø Created manually with the test suite editor or automatically by a recorder. Ø May be associated with a test script or Java class. Ø Executable with TPTP test execution framework. Ø Datapool Ø Provides input and expected output data to a test. Ø Consists of equivalence classes, variables and records. Ø Created with the datapool editor or imported from *. CSV files. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 30

Testing Tools – Basic Concepts Ø Test Execution Ø Framework for local and remote test deployment, execution, results collection. Ø Typically started from Launch Configuration. Ø Automatable Services Framework (ASF) for launching tests programmatically via scripts (e. g. shell and ANT) and external applications (e. g. Java). Ø Test Log Ø Persisted execution results. Ø Test log viewer to summarize, view, navigate and filter execution events, and associated defects. Ø Report Ø Aggregates and summarizes numerous test execution results over a period of time (report window). Ø BIRT integration and extensible report generators. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 31

Testing Tools – Test Types Ø Manual testing of applications by human testers based on textual test descriptions (plain text or HTML). Ø Manual Test View for executing a manual test on the target machine allowing the tester to step through the manual test and capture messages and results: Ø Verdicts Ø Reasons Ø Information Ø Attachments Ø JUnit and JUnit Plug-in Ø Automated unit testing of applications and plug-ins based on JUnit test classes. Ø JUnit test classes generated and synchronized automatically from the test suite while preserving user modifications. Ø Wizard for creating TPTP JUnit and JUnit plug-in tests from existing JUnit and JUnit plug-in test classes without source modification. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 32

Testing Tools – Test Types Ø URL Ø Automated HTTP performance testing of Web applications from JUnit test classes. Ø HTTP proxy recorder to intercept and record HTTP requests/responses from user interactions with Web applications to create an URL test suite. Ø Automated GUI Recorder (AGR) Ø Automated GUI recording and playback for testing Eclipse-based UIs. Ø Positional and object-based GUI recorder to intercept and record user interactions with UI widgets in the workbench to create an automated GUI test suite with datapools and verification points. Ø Object mine and adaptive widget resolution to help maintain tests. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 33

Testing Tools – API Recorder Ø Records the behavior of local and remote Java applications. Ø Captures API invocations and their arguments/return values based on user-defined package, class, and method targets. Ø Recordings are used for analysis, testing (e. g. datapools), and generating TPTP JUnit tests. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 34

Testing Tools - Demo Ø Target API November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 35

Testing Tools - Demo Ø Start an API Recording… November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 36

Testing Tools - Demo Ø API Recording Configuration – Test Suite Generator November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 37

Testing Tools - Demo Ø API Recording Configuration – Target API November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 38

Testing Tools - Demo Ø JUnit Test Suite - Overview November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 39

Testing Tools - Demo Ø JUnit Test Suite – Test Methods November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 40

Testing Tools - Demo Ø JUnit Test Suite – Behavior (invocation) November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 41

Testing Tools - Demo Ø JUnit Test Suite – Behavior (loop) November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 42

Testing Tools - Demo Ø JUnit Test Suite – Test Class November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 43

Testing Tools - Demo Ø Run the JUnit Test Suite… November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 44

Testing Tools - Demo Ø Test Log - Overview November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 45

Testing Tools - Demo Ø Test Log - Events November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 46

Testing Tools - Demo Ø Create a Test Report… November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 47

Testing Tools - Demo Ø BIRT Test Pass Report November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 48

Testing Tools - Demo Ø Create a Datapool… November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 49

Testing Tools - Demo Ø Datapools November 14, 2007 - Overview Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 50

Testing Tools - Demo Ø Datapools November 14, 2007 – Equivalence Class Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 51

Testing Tools - Demo Ø Using the Datapool November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 52

Testing Tools - Demo Ø Help > Help Contents November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 53

Agenda ü ü ü Ø Overview of TPTP Profiling tools TPTP Testing tools What’s new? November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 54

What’s New in TPTP 4. 5 Ø Profiling Tools: Ø Secure profiling (JVMTI) including authentication and encryption. Ø Hover details for method invocations in the Method Statistics view. Ø Simplified stand-alone profiling (JVMTI). Ø Thread contention analysis for locating monitor and data contention. Ø Improved profiling filters (filter sets). Ø Binary data format to increase profiler performance and scalability. Ø Support for Java 6 and IPv 6 networks. Ø Test Tools: Ø Encrypted datapool variables for testing with confidential data. Ø Parallel test execution for launching multiple tests concurrently. Ø Reference maintenance when moving, copying, deleting, renaming, and importing/exporting test assets. Ø URL recording on IPv 6 networks. November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 55

References TPTP home page Ø http: //www. eclipse. org/tptp/ Ø TPTP download page Ø http: //www. eclipse. org/tptp/home/downloads/ Ø TPTP documentation Ø http: //www. eclipse. org/tptp/home/documents/ Ø TPTP tutorials and demos Ø http: //www. eclipse. org/tptp/home/downloads/quicktour/v 44/quick_t our. html Ø TPTP Eclipse corner article Ø http: //www. eclipse. org/articles/Article-TPTP-Profiling. Tool/tptp. Profiling. Article. html Ø November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 56

November 14, 2007 Eclipse Foundation - www. eclipse. org Content provided under the terms and conditions of the Eclipse Public License Version 1. 0 57