Long Desired, It’s Finally Here
The idea of “automatic regression testing” has been considered for many years, but was stymied by several challenges. Those include wiring regression testing into DevOps to create Continuous Regression Testing, and automatically exercising regression testing protocols against all three categories of test (functional, performance, security). But the primary challenge has always been identifying what to test, and then driving the regression testing against that knowledge-base.
Traditionally, testers have created tests based on their understanding of how an app can or should be exercised. This can be an exceptionally large universe of possibilities for any good-sized system with an active user base. Further, users often act in surprising or idiosyncratic ways. After all, another word for users is people, and people are unpredictable.
Fortunately, a source of actual user activity exists — in production logs. However, production logs basically store “breadcrumbs,” mostly for security purposes. To the naked eye, they appear to be a huge jumble. Ask anyone who has had to dig through them! Yet, production logs are a big-data source of actual user activity, so learning from them allows us to assemble a model of what users did, where they went in the app and when. From that model, we can create regression tests that mimic the actions of thousands or even millions of actual users.
This sounds easy enough. However, production logs only record hints of server requests and limited response codes. It doesn’t sound like much to go on.
Enter AI. Yes, this data heap is an obvious place to deploy an expert system to dig through the pile of breadcrumbs and make sense of it all, at a minimum. But, we can also use machine learning to “learn” about the application under test…even if that application has changed somewhat from the production version. By creating a blueprint of the application, and how it works at the UX and the API level, and learning how it makes requests, we end up with a whole picture of the application, its paths and its workings.
To go any further, we would have to learn how the application is validated with session IDs and security tokens, which can occur in a few steps or at nearly every step. This correlation mapping would also have to occur. Fortunately, machine learning can also be deployed against this normally sticky problem.
Plus, every application has legitimate data for credentials and forms, and these must be tied to a database and used appropriately.
Lastly, one would have to develop a language that describes user actions and takes into account virtually any request or response a web-based application might do. Then, create a code generator to create that code, and also a code reader to execute that code, with all of the associated automation and reporting requirements.
If one had all of the above technology, one could then instantly and automatically generate regression tests based on what real users did over the course of weeks or months.
The algorithms and code to accomplish this seem daunting, to say the least. And they were.
Fortunately this is now commercially available within Appvance IQ. It is next-gen regression testing, since it represents 100% test coverage of what users actually do with an application, generated automatically…so you don’t have to. No clicks to record and no code to write. It just simply works.
Further, Appvance IQ’s automatic regression testing integrates with DevOps tooling such as Bamboo and Jenkins to enable Continuous Regression Testing, and is a unified testing system, so all three categories of test (functional, performance, security) can be automatically executed.
If you have not seen this in action, grab a 90 second video demo here.
If you have a budget and are ready to dive in, request a personal live demo here.
