Use of eVgen Feature to Test A Combinatoric Application

A possible solution to testing a web application with apparent high-count combinations of input may be to use a continual testing process with randomly chosen values for each test case.

For a very large permutation count you will never try every combination. But over time, as the total number of runs increases, and the total number of combinations tried increases, you will be able to accumulate sufficiently many successful runs to generate some confidence in the integrity of your application.

Here is how this could work in practice.

• Create your test starting with an eValid recording of your web application behavior.
• Parameterize that script with variables whose values are to be assigned at the start of each test.
• Construct a table of possible values for each parameter.
• Use eVgenRand (an eValid suite component) to create the values for each instance of playback using the "random selection" option, and specify a large number of tries to make.
• Make sure errors are recorded somehow for later study.

If tests each take one minute, after a week you have about 10,000 tries and some number, N, of tests that fail. Then:

• If N continues to be zero, then you can increased confidence in the quality of you web application. The more you run the better you feel.
• For each of the N failures you have hard evidence of a problem that needs to be fixed.

This may not be as efficient as proposed schemes from the R&D community that attempt to reduce combinatoric complexity from the outset. But this solution has the advantage of easy implementation, and immediate availability. The relevant eValid commands involved are described in the eVgen Example: Yahoo! Random Search example.