In the upcoming Linux 4.14-rc3 release, work continues to develop the Kselftest TAP13 framework API and convert tests to TAP13. The new tests include Kselftest common RUN_TESTS in lib.mk that have been enhanced to print TAP13 to cover test shell scripts that won’t be able to use the Kselftest TAP13 API; this also covers test programs that aren’t converted yet.
Several fixes have been made to existing tests to prevent failure in unsupported cases as part of an ongoing work based on feedback from Kselftest stable release users that don’t want the tests to fail due to unmet dependencies, such as config options being disabled.
Additionally, a new watchdog test has been added and much needed cleanups to the existing watchdog tests have been made by Eugeniu Rosca.
A New Kselftest Use-Case
A notable change in this release is new support for the “make O=dir kselftest” use-case. Several developers rely on this use-case to relocate objects and keep their source tree clean. It comes in handy when developers want to build kernels for several architectures and/or with different config files from the same source tree and keep it clean. Objects for each of those builds are saved in separate object directories.
I added support for this use-case by extending the existing Kselftest KBUILD_OUTPUT use-case. The majority of changes have been made to the kselftest common infrastructure. Some test make files are changed as needed to address the custom build and run_tests.
With this change the following methods to build and run Kselftest are possible.
Build all, relocate objects to /tmp/kselftest, and run tests
make O=/tmp/kselftest kselftest
make KBUILD_OUTPUT=/tmp/kselftest kselftest
Build TARGETS, relocate objects to /tmp/kselftest, and run tests
make O=/tmp/kselftest TARGETS="futex sync size" kselftest
make KBUILD_OUTPUT=/tmp/kselftest TARGETS="futex sync size" kselftest
make O=/tmp/kselftest kselftest-clean
make KBUILD_OUTPUT=/tmp/kselftest kselftest-clean
Better Test Summaries
One other important change is that messages from the Makefile are silenced during the test run, and the individual test results have been redirected to /tmp/testname files. This change makes it easier to understand the test results summary, while still being able to look at the detailed test output.
For example, running net tests after this change will output just the summary:
make --silent TARGETS=net kselftest TAP version 13 selftests: reuseport_bpf ======================================== not ok 1..1 selftests: reuseport_bpf [FAIL] selftests: reuseport_bpf_cpu ======================================== not ok 1..2 selftests: reuseport_bpf_cpu [FAIL] selftests: reuseport_bpf_numa ======================================== not ok 1..3 selftests: reuseport_bpf_numa [FAIL] selftests: reuseport_dualstack ======================================== not ok 1..4 selftests: reuseport_dualstack [FAIL] selftests: reuseaddr_conflict ======================================== not ok 1..5 selftests: reuseaddr_conflict [FAIL] selftests: run_netsocktests ======================================== ok 1..6 selftests: run_netsocktests [PASS] selftests: run_afpackettests ======================================== ok 1..7 selftests: run_afpackettests [PASS] selftests: test_bpf.sh ======================================== not ok 1..8 selftests: test_bpf.sh [FAIL] selftests: netdevice.sh ======================================== ok 1..9 selftests: netdevice.sh [PASS] selftests: rtnetlink.sh ======================================== ok 1..10 selftests: rtnetlink.sh [PASS]
The detailed test results can be found under /tmp for each test:
reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa reuseport_dualstack reuseaddr_conflict run_netsocktests run_afpackettests test_bpf.sh netdevice.sh rtnetlink.sh
Please check the Kselftest documentation for supported use-cases: Documentation/dev tools/kselftest.rst