Testing with CTest and CDash Julien Jomier Kitware

Testing with CTest and CDash Julien Jomier Kitware Inc. National Alliance for Medical Image Computing http: //na-mic. org

Testing Overview Developers CVS/SVN Web Browser CTest CDash National Alliance for Medical Image Computing http: //na-mic. org

Testing Terminology Sites Builds dash 5. kitware Linux-g++ Linux-gcc 4 iris. elmtech Linux-gcc 3 National Alliance for Medical Image Computing http: //na-mic. org Dashboard

Why Test Daily? • Large code base too large for any single developer to understand • Developers distributed around the world • Identify problems as they occur • Insure that object API remains unchanged • Provide feedback to developers as they experiment with new implementations National Alliance for Medical Image Computing http: //na-mic. org

“If it’s not tested it doesn’t work” • Dasboards and Software Process – Roots in GE Six Sigma initiative • Cross platform testing • Nightly regression testing Memory (Purify/Val. Grind) testing • Coverage testing • Style checking • Documentation evaluation National Alliance for Medical Image Computing http: //na-mic. org

How to write a good test? • Use empirical values (5/2=2. 5) • Test most of the use cases (even the boring/most difficult ones) • Make your tests as efficient as possible • Have someone else write the test for your algorithm National Alliance for Medical Image Computing http: //na-mic. org

Regression Testing • Compare generated image against standard “correct” image – pixel-by-pixel comparison – can use a threshold metric – adjusted for effects like dithering National Alliance for Medical Image Computing http: //na-mic. org

CTest • Testing client • Distributed with CMake • Submit test results to the dashboard • XML submission • Independent of CMake National Alliance for Medical Image Computing http: //na-mic. org

CTest with CMake. Lists. txt INCLUDE(CTest) ENABLE_TESTING() # Add a simple test ADD_TEST(CDash. Test 1 CDash. Test) ADD_EXECUTABLE(CDash. Test main. cxx) CTest. Config. txt set(CTEST_PROJECT_NAME "My. Project") set(CTEST_NIGHTLY_START_TIME "21: 00 EDT") set(CTEST_DROP_METHOD "http") set(CTEST_DROP_SITE "localhost") set(CTEST_DROP_LOCATION "/CDash/submit. php? project=My. Project") set(CTEST_DROP_SITE_CDASH TRUE) National Alliance for Medical Image Computing http: //na-mic. org

Testing Submission Compiler/Tester CDash CTest Backup XML File HTTP PUT XML Submission XML Parsing Update. xml Fill Database Send Email Configure. xml Build. xml Test. xml Coverage. xml Dynamic. Analysis. xml National Alliance for Medical Image Computing http: //na-mic. org Update RSS Feed

CDash • • New generation of Dart Written in Php/Javascript My. SQL Open Source Run on top of a web server XSL implementation Web 2. 0 National Alliance for Medical Image Computing http: //na-mic. org

CDash Features • • • Ajax requests Graphs: build, test times Builds group User management Email notifications Timing defects National Alliance for Medical Image Computing http: //na-mic. org

Coverage Support • Email Low Coverage • Bullseye support National Alliance for Medical Image Computing http: //na-mic. org

Build Notes • Share with other developers • Keep it simple National Alliance for Medical Image Computing http: //na-mic. org

My CDash • • Personal Page Subscribe to project View sites Email preferences National Alliance for Medical Image Computing http: //na-mic. org

Project Roles • Normal user • Site maintainer • Project administrator National Alliance for Medical Image Computing http: //na-mic. org

Claiming Sites • • • Site information Automatic (CTest 2. 6) Information history Manual editing Personal view/email National Alliance for Medical Image Computing http: //na-mic. org

CDash Misc. • Automatic Geolocation from IP addresses • Support for i. Phone National Alliance for Medical Image Computing http: //na-mic. org

Installing CDash • (L/W)AMP : Apache, My. SQL, PHP - XAMPP, Easy. PHP • Download CDash (www. cdash. org) in your Apache web directory • Edit config. php • Point Web Browser to CDash - Setup the database - Create first project National Alliance for Medical Image Computing http: //na-mic. org

Style Checking • • • KWStyle C++ Style checker Configuration via XML HTML Generator (TBI in CDash) Open Source Written in C++ National Alliance for Medical Image Computing http: //na-mic. org

Contribute • Website - http: //www. cdash. org - http: //public. kitware. com/KWStyle • Bug Tracker - http: //www. cmake. org/Bug • Mailing List - http: //public. kitware. com/cgibin/mailman/listinfo/cdash National Alliance for Medical Image Computing http: //na-mic. org

Happy Testing! National Alliance for Medical Image Computing http: //na-mic. org