The benefits of AI-driven testing go well beyond automatic test-script generation, profound and game-changing as that is. However, auto test-script generation is robustly covered elsewhere on this blog, so this post introduces two downstream benefits of AI-driven testing: Intelligent Test Prioritization and Test Results Analysis.
AI-aided test prioritization and results analysis are each transformative in themselves, so let’s explore what they include and the benefits they provide.
Intelligent Test Prioritization
Intelligent Test Prioritization has two areas of benefit. First, it allows a testing team to direct its focus to the top 10% of use cases, knowing that the AI-driven testing will take care of the rest. In fact, the AI-driven testing will cover the top 10% of cases as well, so the application of human testing to those areas effectively creates double-coverage where it matters the most. This is very reassuring for areas of the application-under-test (AUT) where failure is not an option.
Second, intelligent test prioritization allows the AI to be trained to go deep into important functions, e.g., transaction processing, even as it is trained to avoid areas where there is nothing of value to test, e.g., marketing, legal.
AI-driven Test Results Analysis
Appvance IQ creates an AI-generated blueprint of the AUT, and then operates from that blueprint. This includes a Blueprint Coverage Map which provides powerful insights into how the AI is following its training and what coverage it is achieving.
The Blueprint Coverage Map aides test results analysis by answering three important questions.
- Where do we need to train the AI to go deeper? There might be areas of the AUT that can be further plumbed by the AI. Identifying those areas and then training the AI to map and test them stems from this analysis.
- Which areas might we want the AI to avoid? Even though AI-driven testing is essentially free, there is no need to have it retest static and/or boilerplate areas, e.g., static marketing or legal pages. This is especially true as one gets closer to the release date where it is imperative to get a full regression of the code after every build delivered to the test environment.
- Where are the errors? The Blueprint Coverage Map and its attendant results reporting show where errors are clustering and aides in root-cause analysis.
For instance, it might show that errors are clustering with some specific third-party code, or in an area that came from an inexperienced dev team. This is invaluable feedback for the dev manager.
Error tracking isn’t limited to functional errors. It also includes performance errors. For instance, are performance glitches clustered with external endpoints? Or perhaps a poor network is causing performance issues. As with identification of functional error clustering, visibility into performance error clustering provides invaluable feedback for the dev manager.
Summary
Test prioritization and results analysis have been transformed by AI-driven testing almost to the same degree that test creation has. While they don’t transform what was once a labor-intensive activity into a labor-free one, they do dramatically reduce the work required for prioritization planning and test results analysis. Plus, they enable much more effective prioritization and results analysis.
The combined benefit greatly improves the quality of an AUT even as the labor required for that achievement is minimized and precisely targeted.
To see AIQ in action, schedule a customized demo with the Appvance team.
