Metaivi Testing Bo F October 12 2017 Status

Meta-ivi Testing Bo. F October 12, 2017 | Status, How-to and futures Tolkien Joh Stephen Lawrence Senior MTS, Wind River (Yocto Baseline Maintainer) Principal Engineer, Renesas (BIT Lead) This work is licensed under a Creative Commons Attribution-Share Alike 4. 0 (CC BY-SA 4. 0) GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries. Copyright © GENIVI Alliance 2017.

Agenda • Status of public baseline testing • How-to on building, running and extending meta-ivi-test • Automation and next steps

Status / where we are

Yocto baseline test framework (meta-ivi-test) • History – Historically release testing took place within members • EGs tested their s/w on (0. 1, 0. 2) pre-releases and report result to BIT – meta-ivi-test layer added • Source, wiki • From meta-ivi 11 (Miranda) by Tolkien (Wind. River) • Release test results for recent GENIVI releases posted on wiki • Provides an image for testing – Created by baseline test build, not release build – Based on some debug/test codes of GENIVI s/w component • Mainly component unit tests

Yocto baseline test framework • Pros – Simple • Cons – Maintained by one person • It does not have all tests from all GENIVI components • We need help/contribution to improve coverage and add missing component unit tests – There is no generated report • It provides some “raw” script terminal output only • To integrate into CI or be used for JIRA more easily we may have to convert this to some formatted text like XML, JSON

Meta-ivi-test How-to

Meta-ivi-test source layout /opt/tests/ Test runner Test Set test-fw. sh audiomanager/ audiomanager_t. inc Test case runner Am. Control. Interface. Test Am. Router. Test Am. Telnet. Server. Test Am. Map. Handler. Test Am. Routing. Interface. Test capicxx-perf/ vsomeip/ Am. Router. Map. Test case

How to use meta-ivi-test • Tests are run in the meta-ivi runtime environment • We will cover: – Building and booting a test-image – How to create tests • Create a test set • Create a test case

Building and booting a test image • add meta-ivi-test layers to <build dir>/conf/bblayers. conf BBLAYERS += " ##your source code root##/poky/. . /meta-ivi-test " • build test-image export TEMPLATECONF=`pwd`/meta-ivi/conf source poky/oe-init-build-env build MACHINE=qemux 86 -64 bitbake test-image • Boot test-image (qemu use case). . /meta-ivi/scripts/runqemu test-image qemux 86 -64 nographic • For real h/w case deploy test-image to h/w and boot

Test runtime commands • run all tests of all s/w root@qemux 86 -64: ~# /opt/tests/test-fw. sh ##### audiomanager ##### [ number of tests: 7 ] [ run_1 ][ PASSED ] 19 tests. . • list the supported components and the number of tests they provide root@qemux 86 -64: ~# /opt/tests/test-fw. sh list audiomanager 7 audiomanagerplugins 5 capicxx-perf 1 common-api-test 1 <s/w name> <number of test cases> dlt-daemon 2. . .

Test runtime commands (cont) • run the test set for a specific component root@qemux 86 -64: ~# /opt/tests/test-fw. sh dlt-daemon ########## [ number of tests: 2 ] [ run_1 ] PASSED [ run_2 ] PASSED • run an individual test case root@qemux 86 -64: ~# /opt/tests/test-fw. sh dlt-daemon 1 ##### dlt-daemon ##### [ run_1 ] test. File 1 Run 1 successful test. File 1 Run 2 successful test. File 2 Run 1 successful. . .

Creating a test set • Create directory for test set <Test. Set> root@qemux 86 -64: ~# mkdir /opt/tests/Sample root@qemux 86 -64: ~# ls -F /opt/tests/ Sample/ node-health-monitor/ audiomanager/ node-startup-controller/ audiomanagerplugins/ node-state-manager/. . . • Create test set runner - filename should be /opt/tests/<Test. Set>_t. inc root@qemux 86 -64: ~# touch /opt/tests/Sample_t. inc root@qemux 86 -64: ~# chmod +x /opt/tests/Sample_t. inc root@qemux 86 -64: ~# ls -F /opt/tests/Sample_t. inc*

Creating a test set (cont) #!/bin/sh get_list() { return 1 } get_title() { echo “Sample" } _run_dir=/opt/tests/Sample run_1() { if [ "x${1}" != "x" ]; then ${_run_dir}/hello >& /tmp/t _c 1=$? if [ "x${_c 1}" = "x 0" ]; then echo " PASSED" else echo " FAILED" fi else ${_run_dir}/hello fi } Script has to provide the following three functions: get_list() -> return the number of testcases get_title() -> return the string of test title actual test function. The name must be run_x() like run_1() run_2(). . . If argument 1 is exist, please, reduce log messages

Creating a test case • C example #include <stdio. h> #include <string. h> int main(int argc, char *argv[]) { if ((argc > 1) && (strncmp(argv[1], ”mute”, 4) == 0)) return 0; printf(”hello, worldn”); return 0; } • Shell script example #!/bin/bash arg=$1 if [ ”x${arg}” = ”xmute” ]; then exit 0 fi echo “hello, world” exit 0

Support • Please post questions to meta-ivi mailing list

Where we want to go

What is next? • Complete Genivi component coverage – Welcome contributions to add unit tests • Integrate with Go CI server – https: //go. genivi. org/ • Automation – Fuego seems attractive • In heavy use in AGL (share test cases) and elsewhere. • Plan to integrate Fuego into CI server • Then add meta-ivi-tests into Fuego – Other options not shut out • LAVA • Kernelci • Who can help?

Thank you! Visit GENIVI at http: //www. genivi. org or http: //projects. genivi. org Contact us: help@genivi. org This work is licensed under a Creative Commons Attribution-Share Alike 4. 0 (CC BY-SA 4. 0) GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries. Copyright © GENIVI Alliance 2017.