Lustre User Group Austin TX April 2012 Whamcloud

Agenda • Whamcloud’s View Of Quality • A Year’s Progress • The Year Ahead

Whamcloud’s View Of Quality 3 © 2012 Whamcloud, Inc.

Whamcloud’s View Of Quality Eric vs. Non Delivery ‟ The only thing I’m non-Linear

Whamcloud’s View Of Quality il ab St F Pe eat rfo ur rm es

Whamcloud’s View Of Quality • Whamcloud is investing money, time and expertise in Quality

A Year’s Progress 7 © 2012 Whamcloud, Inc.

Landing Test Performance • For landing testing we have carried out; – – 687

Landing Test Performance • This chart shows the percentage of landings that passed all

Distributed Test ISVs Source code repository Public Sector Resellers 10 © 2012 Whamcloud, Inc.

Distributed Test But! ISVs Source code repository Public Sector Resellers 11 © 2012 Whamcloud,

Juelich Supercomputer Centre • Juelich financed a fully equipped cluster • Used for testing

Indiana University • Indiana provided a 36 node cluster for the development and rollout

The Year Ahead 14 © 2012 Whamcloud, Inc.

Development vs. Landing Test • Which is landing test? • Which is development test?

Development vs. Landing Test • Distinct paths for development vs. landing – Similar process

Development Test Cloud 17 © 2012 Whamcloud, Inc.

The Goal Landing Results 100% 80% 60% Fail 40% Pass 20% 100% 0% 80%

Summary • Whamcloud’s Quality Approach • A Review Of The Last Year • Our

Maloo ‘Your Window onto Test’ 20 © 2012 Whamcloud, Inc.

Maloo Agenda • What is Maloo? • Recent changes to Maloo – High points

What is Maloo? • A repository for lustre test result data – Collects the

Recent changes to Maloo 23 LUG Austin, TX - April 2012 © 2012 Whamcloud,

Release report https: //maloo. whamcloud. com/reports 24 LUG Austin, TX - April 2012 ©

Node utilization report https: //maloo. whamcloud. com/reports/show_node_utilization_report 25 LUG Austin, TX - April 2012

Internal changes • Invisible work – – – 26 Development practices Testing Packaging Deployment

Development priorities 27 LUG Austin, TX - April 2012 © 2012 Whamcloud, Inc.

Development priorities • Tool quality – Mechanics of Maloo and its development – Visibility

Development priorities • Data quality – Accurate – Timely – Accessible 29 LUG Austin,

New features 30 LUG Austin, TX - April 2012 © 2012 Whamcloud, Inc.

New features Log file scanning • Automates a tedious task 31 LUG Austin, TX

Maloo Footprints 33 © 2012 Whamcloud, Inc.

Maloo Footprints 34 © 2012 Whamcloud, Inc.

Maloo Footprints 35 © 2012 Whamcloud, Inc.

New features User profiles • One size needn’t fit all 36 LUG Austin, TX

Wrap up and questions What was covered • Changes in Maloo since last year

Thank you very much 38 © 2012 Whamcloud, Inc.

Slides: 38

Download presentation

Whamcloud’s View Of Quality • Whamcloud is investing money, time and expertise in Quality • …is continuing to develop tools and invest in infrastructure to enhance the Lustre community • The whole of its engineering team is orientated around quality principles 6 © 2012 Whamcloud, Inc.

Landing Test Performance • For landing testing we have carried out; – – 687 days of landing testing 2381 landing sessions 2092758 individual tests 2087607 passed • Statistically – 1425 passes – 956 failures – 59. 85% Session passed 8 © 2012 Whamcloud, Inc.

Landing Test Performance • This chart shows the percentage of landings that passed all tests since LUG 2011 80, 00% 75, 00% 70, 00% 65, 00% 60, 00% 55, 00% Pass Rate 50, 00% Linear(Pass Rate) 45, 00% 40, 00% 35, 00% 05. 2 01 1 06. 2 01 1 07. 2 01 1 08. 2 01 1 09. 2 01 1 10. 2 01 1 11. 2 01 1 12. 2 01 1 01. 2 01 2 02. 2 01 2 03. 2 01 2 30, 00% 9 © 2012 Whamcloud, Inc.

Juelich Supercomputer Centre • Juelich financed a fully equipped cluster • Used for testing all head releases • Specialises in failover testing – 36 TB of multi attached storage • Good for performance regression tests – Private network with no contention so results are repeatable • Also used for manual large Lun testing for 2. 2 release • I’d like to thank Juelich and in particular Frank Heckes for making this happen 12 © 2012 Whamcloud, Inc.

Indiana University • Indiana provided a 36 node cluster for the development and rollout of a backup test system • Used for secondary 2. 2 release testing • Enabled the transfer of the primary test cluster from the West Coast to Colorado without any break in the Lustre test and landing processes • Again I’d like to thank Indiana and Steve Simms for enabling this 13 © 2012 Whamcloud, Inc.

Development vs. Landing Test • Which is landing test? • Which is development test? 100, 00% 90, 00% 80, 00% 70, 00% 60, 00% 50, 00% Pass Rate 40, 00% Linear(Pass Rate) 30, 00% 20, 00% 10, 00% 05. 2 01 1 06. 2 01 1 07. 2 01 1 08. 2 01 1 09. 2 01 1 10. 2 01 1 11. 2 01 1 12. 2 01 1 01. 2 01 2 02. 2 01 2 03. 2 01 2 0, 00% 15 © 2012 Whamcloud, Inc.

Development vs. Landing Test • Distinct paths for development vs. landing – Similar process but distinct purpose • Provide for maximum flexibility in development testing – Encourage systematic test to be part of the development process – Test during development does improve product quality • Encourage peer review as the code is written – Earlier review leads to better code and more opportunity for education • Be auditable – Improvement requires knowledge of the past – Development test part of the landing collateral • 100% pass rate for landing test – Developers should push tried and tested code for landing 16 © 2012 Whamcloud, Inc.

Maloo Agenda • What is Maloo? • Recent changes to Maloo – High points since last LUG • Development priorities – Tool quality – Easy access to timely, accurate data • Some planned features – User preferences – Automated scanning of incoming log files for “interesting” data 21 LUG Austin, TX - April 2012 © 2012 Whamcloud, Inc.

What is Maloo? • A repository for lustre test result data – Collects the test results and the logs generated – Allows users to query the database – Contains about 1 TB of log files • https: //maloo. whamcloud. com 22 LUG Austin, TX - April 2012 © 2012 Whamcloud, Inc.