Monday, November 26, 2012

What makes a browser test enabled?

The other day someone made this observation: "You say your browser is 'test enabled.' But what exactly does that mean?" Considering that this is a main architectural feature of the eValid product, the centerpiece of the eValid Test Suite, it's important to understand why we use that terminology.

About Experiments
The idea of any kind of testing is rooted in the notions of experimental science: Known inputs applied to an "application under test (AUT)" produce results that imply facts about the AUT.

The isn't different whether the what's being tested is hardware or software. The Software Testing process for of a web browser enabled application typically involves applying a Test Method through a Testbed that permits application of Test Data to the AUT -- the web application itself is the AUT. Each Software Test constitutes one instance of a scientifically oriented Experiment the purpose of which is to accomplish successful Verification and Validation of the web application.

How Do You Test Software?
To test software, you need a place to run it -- to feed it data and make observations about what happens when you do. That is, a Testbed.

An obvious choice for a testbed for a web browser enabled application is to use a Web Browser as the launch-point for individual tests in the experiments performed to establish web application quality.

OK, you say, great. But how do you make a browser into a testbed for software applications? Every browser will test web applications: you type, you see, you can even take a screenshot. How can you mechanize that?

Browser Architecture Exploited by eValid
This is where eValid is different from regular browsers. The inventive thing we did is craft the test enablement using the components of a regular browser. eValid has a separate script recording and script playback engine that is built into the browser executable, but this is done in a way that does not change the way the browser works (so the results are accurate), and does not modify how JavaScript operates inside the browser (so the test browser can continue to run complex AJAX applications).

The details, of course, can be complicated but eValid makes use of resources that every browser automatically has built in. The main resource that eValid uses is Document Object Model that provides direct and immediate facts about a page that has been read and rendered into the browser face. Having this resource available makes it easy and straightforward to validate and verify a web application.

The conceptually simple approach that eValid takes provides a number of very important practical advantages. The main one is verisimilitude -- that the tests run by the eValid browser with its testing augmentation are as similar as possible to what happens with an actual browser. The advantages also include low overhead, ability to extract accurate timings and component byte counts, and the ability to assure test synchronization using non-interfering methods that interrogate the DOM.

How It Plays Together
From a scientific point of view let's see how the eValid architecture shapes up compared with a conventional experimental setup:

Scientific Experiment eValid Test Perspective
The set of hypotheses, the test data, you give to the test object that will illustrate the needed behavior. The eValid script with its instructions on how to drive the browser through the application.
The testbed setup reveals the results of application of the test data. The resulting rendered page(s) that the input script directs the browser to visit.
The measurements from the test object that are used to verify the hypotheses are true (or are not). The set of validations or checks that determine whether the test PASSes or FAILs.

 In short, the eValid testbed system really is a Test Enabled Web Browser.

No comments: