Abstract:
In the 3.0 Zeus release, we have added ‘resulttool’ and ‘yocto-testresults’. This new age of enlightenment gives us much more data about how the builds are proceeding and a measure of the health of each of the components. What is resulttool? How do we interpret the results? What can we do with those results?
Embrace the power of the Force and realize that we can now perform Anomaly Detection on the results to catch regressions that a Human needs to pay attention to. Do humans need to look at every TestRun and every TestResult? NO. One thousand times NO. With some automation, we can ingest the testresults and perform Anomaly Detection on the results. We can then flag egregious offenders to the historic results and highlight the differences. This allows us to repurpose high-value QA resources to investigate regressions and unexpected results, rather than manually testing or manually investigating test results. In this presentation, we discuss some pre-release approaches to ingesting the test cases and test results. We also discuss some approaches to algorithmically detect significant changes (regressions) in the results over time.
Speaker: Tim Orling, Sr. Linux Software Engineer, Intel Corporation
Tim Orling is a software engineer at the Intel Open Source Technology Center. Tim joined Intel in early 2016 after many years as a volunteer developer for OpenEmbedded and the Yocto Project. He has been an open source software and embedded hardware enthusiast for many years. He taught in a university setting for more than 5 years and has given many technical talks at conferences.