Test Driven Development Behavior Driven Development John Diamond

Test Driven Development & Behavior Driven Development John Diamond, AD/Instrumentation 4/27/2020

What is the agenda? • • • 2 Motivation: Why Test Driven Developer and Behavior Driver Development? TDD overview Unit Test and Code Coverage Examples from Mu 2 e. ER BDD & Cucumber overview Cucumber examples from Mu 2 e. ER Q&A John Diamond | TDD & BDD 4/27/2020

Remember the CI/CD policies? 1. Every artifact related to the design, construction and operation of the software system should be under version control 2. Commit early, commit often, limit branches 3. Don’t commit anything that breaks the build 4. Every project and it’s dependencies should be tracked by the CI server 5. Every project should include a unit-test suite that achieves at least 80% coverage 6. Every software project should include an easy to use, automated deployment and roll-back process 7. Every project should maintain a test environment that mirrors the production environment as much as possible 8. Every project should include an acceptance test-suite that verifies the ‘business’ goals are being met 9. Manual tests should be documented and tracked by the CI server 3 John Diamond | TDD & BDD 4/27/2020

Why practice TDD and BDD? • Test-Driven Development (TDD) – Helps you develop a unit test-suite that covers all of your code • Behavior Driven Development (BDD) – Helps you communicate with your users, gather and agree on requirements AND develop an acceptance test-suite • Complementary practices, not mutually exclusive – Construction & Design: use TDD • “Build the thing right” – Analysis & Design: use BDD • “Build the right thing” 4 John Diamond | TDD & BDD 4/27/2020

What is Test Driven Development (TDD)? • References – Test Driven Development by Example (Kent Beck) – Test Driven Development for Embedded C (James Grenning) • • My quick definition: write your tests before you write your code The TDD micro-cycle (K. Beck): – – – 5 Add a small test Run all tests, see the new test fail (maybe fail to compile) Make small changes that make the test pass Run all tests, see the new test pass Refactor to remove duplication and improve expressiveness John Diamond | TDD & BDD 4/27/2020

What is a unit test? • An automated test at the class method / function level – Tests one feature / one path through the method • Tests should be stateless and should not depend on each other or their execution order – Frameworks provide hooks for initialization and tear-down of tests • Tests run inside of a testing framework – Provides a harness so you’re only writing code that performs the test – Provides macros for testing conditions and reporting failures back to the framework 6 John Diamond | TDD & BDD 4/27/2020

How does this work for embedded projects? • Embedded development has some testing challenges… – Host & Target architectures are usually different – Use two-target approach • Application builds for both host and target architecture • Unit tests run in the host environment during development for immediate feedback – Most of the time you run ‘make tests’ right after a ‘make’ – Quick feedback is key so you can find what’s broken and fix it right away • Unit tests also run in the test environment during testing and in the CI/CD pipeline – Getting to the target test environment takes more time / several steps – Can not provide immediate feedback – What about hardware? • Develop Stubs and Mocks to stand in • Eliminates HW bottleneck: allows development to proceed before hardware is ready • But it also has some advantages… – No User Interface – ACNET Interfaces • Testing through ACNET is more acceptance-level testing(will cover later in BDD) • But do test your interface code – Unit test around the cdev Device methods – Coverage reports help with edge conditions here 7 John Diamond | TDD & BDD 4/27/2020

How do I know I am writing enough tests? • • Each project is different – aim for a reasonable amount of test coverage Test coverage is the amount of code executed during your unit test run – 90 -95% should be achievable for embedded / front-end projects • Coverage reports can tell you how much, show you what code is not being covered – This can guide you to what kind of tests you should write • 8 Test coverage should be a requirement in the CI/CD pipeline John Diamond | TDD & BDD 4/27/2020

What tools should I use for TDD? • • Lots of testing tools out there, no clear winners (as usual) Mu 2 e Slow Extraction front end (embedded C/C++ application) – Unit test framework: Cpp. Unit • One of two recommended frameworks in Test Driven Development for Embedded C (J. Grenning is one of the developers) – Unity is the other • Similar to other x. Unit frameworks • Works with C and C++ • https: //cpputest. github. io/ – Code coverage reports: Gcov • Standard w/ GCC • https: //gcc. gnu. org/onlinedocs/gcc/Gcov. html • Web-based reports w/ lcov – (Haven’t figured out how to make this work w/ multi-level source tree yet) 9 John Diamond | TDD & BDD 4/27/2020

How does the unit-testing framework (Cpp. Unit) work? • Make a unit. Tests. C to hold tests for your module or class – Contain tests, test groups and support code for that module/class – Tests test one specific function / feature / scenario of a function or method – Test Groups let you associate common initialization and tear-down code • Make a All. Tests. C file for your application or library – Contains a main which runs all of the tests 10 John Diamond | TDD & BDD 4/27/2020

How does the coverage report / Gcov work? • ‘make tests’ runs the unit-test suite, then coverage report. • Reports coverage for each source file • Reports total coverage for all source files at end • Produces *. gcov report for each source file 11 John Diamond | TDD & BDD 4/27/2020

Does unit-testing test everything? • Unit-testing works on the class/method and module/function level – This verifies that software module works as the programmer expects – This does not tell us anything about what the software does as a whole • Acceptance testing works on the functional level – This verifies that the software as a whole works as the user expects – Usually the domain of manual testing… – But this can be automated as well 12 John Diamond | TDD & BDD 4/27/2020

What is Behavior Driven Development (BDD)? • References – The Cucumber Book: Behavior-Driven Development for Testers and Developers (Matt Wynne and Aslak Hellesoy) • • Takes the same “outside-in” approach to writing software as TDD BDD Cycle: – Write a feature specification in a BDD Domain. Specific Language (Gherkin) • The feature specification is communicated with the users • The DSL tool (Cucumber) converts the specification to an automated acceptance test – See the test fail – Write code to make the test pass – Refactor & communicate with users 13 John Diamond | TDD & BDD 4/27/2020

What is Cucumber? • • https: //cucumber. io/docs/guides/ A BDD tool for writing self-executing feature specifications – Command-line tool, works similar to unit-testing frameworks – Written in Ruby, popular with Ro. R projects • Feature specifications are written in a Domain-Specific Language (DSL) called Gherkin – Gherkin uses a Given/When/Then syntax that is highly readable and focused to writing software specifications – Unlike unit tests, feature specifications are meant to be shared and collaborated on with users 14 John Diamond | TDD & BDD 4/27/2020

What do feature specifications look like? • Example from Mu 2 e. ER: • • • First two lines are comments Scenario lines define a test Scenarios have a Given / When / Then structure – ‘Given’ states a precondition for the test – ‘When’ is an action performed by the user – ‘Then’ defines how the system should react to the action 15 John Diamond | TDD & BDD 4/27/2020

How does Cucumber execute the feature specification? • • • Feature specs live in a features/ folder for your project When you run ‘cucumber’ every feature in the features/ fold is executed Cucumber uses Ruby and Regular Expressions to match phrases in the spec, extract parameters and interact with the system – Ruby code is stored in the step_definitions/ folder – Support code goes in the support/ folder 16 John Diamond | TDD & BDD 4/27/2020

But do I have to use Ruby? • No, Cucumber supports many other languages for step definitions – However, I am finding that I like Ruby for this use case 17 John Diamond | TDD & BDD 4/27/2020

Is this the end? • • Still learning about TDD, BDD, Cpp. Unit, Gcov and Cucumber Some goals: – – Explore other x. Unit frameworks for C/C++ (Google? ) Support web-based coverage reports with lcov Add a minimum coverage test to the CI/CD pipeline Use Cucumber and ACL to test Mu 2 e. ER through ACNET • Ruby installed on clx 50 • 18 Questions? John Diamond | TDD & BDD 4/27/2020