Interpreting CTS results

Interpreting CTS results

The CTS test results are placed in the file:

$CTS_ROOT/android-cts/repository/results/<start_time>.zip

If you have built the CTS yourself, $CTS_ROOT will resemble the path out/host/linux-x86/cts but differ by platform. This reflects the path where you have uncompressed the prebuilt official CTS downloaded from this site.

Inside the zip, the testResult.xml file contains the actual results—open this file in any web browser (HTML5 compatible browser recommended) to view the test results. It will resemble the following screenshots.

Note: The results are provided to help you ensure the software remains compatible throughout the development process and act as a common format for communicating the compatibility status of your device with other parties.

Figure 1. CTS test summary

If testResult.xml displays a blank page when using the Chrome browser, change your browser configuration to enable the --allow-file-access-from-files command line flag.

The Device Information section provides details about the device, firmware (make, model, firmware build, platform), and device hardware (screen resolution, keypad, screen type). To access device information, click the link above Test Summary.

The Test Summary section provides executed test plan details, like the CTS plan name and execution start and end times. It also presents an aggregate summary of the number of tests that passed, failed, timed out, or could not be executed.

The next section also provides a summary of tests passed per package.

Figure 2. CTS test report

This is followed by details of the the actual tests that were executed. The report lists the test package, test suite, test case, and the executed tests. It shows the result of the test execution—pass, fail, timed out, or not executed. In the event of a test failure details are provided to help diagnose the cause.

Further, the stack trace of the failure is available in the XML file but is not included in the report to ensure brevity—viewing the XML file with a text editor should provide details of the test failure (search for the <Test> tag corresponding to the failed test and look within it for the <StackTrace> tag).