Ask yourself “what would I do differently if my test team were one thousand people strong and all as good as my very best automation engineer?” That is the team you are about to lead.
Software testing has entered a new era with the arrival of GenAI. But, GenAI’s manifold benefits only come after properly orienting the testing team to this game changing technology and adjusting your testing processes accordingly. So, here are four actions that will get your test operation ready for GenAI-driven test projects.
1. Change the approach to automation.
Shift your focus from human-created automation to AI-created automation. This includes rethinking how to document the testing we want to do (from test cases to business rules), how to recruit and organize the testing team (fewer manual testers, more domain experts), and how to build a test plan.
Test plans should now have a new definition of DONE. Specifically, how we measure automation projects changes now that test coverage is shown to be, like the king with no clothes, rather more uncovered than covered. Further, the software team must prepare for more reported issues than ever before, and for seeing quality from a 360° perspective (not just functional testing but non-functional too).
2. Identify what humans and what the AI will automate.
Specifically, humans should test business critical functions where there is financial, reputational and maybe even criminal risk. AI should test everything else.
This segmentation has several benefits. First, it utilizes test engineers where their judgement, experience and domain knowledge are best utilized, even as it largely eliminates mind-numbing testing from their workload. Second, it optimizes for the best use of scarce and specialized test engineers on the one hand, and newly capable GenAI-powered automation on the other. Last but not least, it ensures complete testing coverage, coupled with a low risk of defects for business-critical functions.
3. Structure test cases to be easily understood by the AI.
To take even the most casual of human specifications and implement them as test cases, we should apply some formal structure to them. We might say, in English, “then hit the big green button” and we might write “click the Submit button”, but in our definition of the step it might be better to say “click the green button under the display field Order Total labelled ‘Buy Now’” to remove any ambiguity. Remember, if the AI is going to translate 1,000 test cases into 1,000 scripts in under 5 minutes, we’re not going to have time to verify them all has having 100% fidelity between Test Case and Test Script.
4. Abstract business rules into AI behaviors, validations, and data entry activities.
To train our AI, we need to teach it three things: how to behave (see this: do this rule), what to validate (see this: test this rule), and what data to use (see this: enter this rule). That means that the requirements engineer should start to think in three dimensions.
By way of comparison, when we define test cases today, we say “do A, do B, do C, get D, check E”, because test scripts are linear. However, the AI operates in three dimensions: every element, every action, on every pathway. It has no time constraints, so needn’t be selective. Unlike human testers, the AI has the capacity to find every element, interact with it in every way, on every possible user pathway.
So, the requirements engineer must think in the abstract as well as in the specific. For instance, on every page where you see quantity, price, and cost, make sure quantity times price equals cost. Whenever you encounter a date-of-birth, make sure it is a valid date and that it is in the past. When dealing with international orders, make sure there is no sales tax.
Now, as the AI traverses the universe of user flows, it is on the lookout for every occasion to test its training. The result is we can write a rule once and it will be deployed thousands of times.
Conclusion
GenAI-powered testing platforms, like Appvance IQ (AIQ), are game changing for testing teams, the software teams of which they are a part, and for the entire conception of software quality. The four actions defined in this Cheat Sheet will position your testing operation to exploit those changes. The result will be previously unknown levels of productivity, optimal use of available staff, and tremendous professional success for the testing team. Every issue, everywhere, every build, every day.
