eValid As An Abstract Model: An Initial Analysis

A suggestion was made some months ago to work out how eValid "implements an abstraction". The idea of this is that any complete technology that addresses a class of problems solves it by lifting an abstraction from the problem class and solving that.

Doing that piece of abstraction analysis is a pretty tall order, and we are not finished with it. But we did recently complete a description of how eValid represents an abstract model that is the based on viewing eValid from its script language perspective.

As our Abstract Model of eValid: Script Language View points out, you can understand eValid quite well as a controllable browser upon which you can take particular actions to produce particular effects. We'll publish more of this set of descriptions as they become available.

How To Test AJAX Applications

There was an interesting post by Markus Clermont and John Thomas on the Google Testing Blog titled Taming the Beast (a.k.a. how to test AJAX applications) that got us to thinking about how much distance there is between what eValid can do and what the community needs -- for testing AJAX in particular. Here is the response we posted:

eValid's Response:
Interesting post, and yes we certainly agree that AJAX web applications can be quite difficult to test. But the picture you seem to paint is rather bleak, and it need not be.


The situation for testing an AJAX application is actually easier than you may think, provide that you have the right tools and by using their underlying technology base can "do the right things" at the right time.

• To overcome the expense of creating functional tests it makes sense to have a good test recorder, which takes away most of the pain of creating tests.
• Tests can be brittle, so you should have some kind of functional test playback feature than automatically adapts the bahavior of the playback engine to overcome inconsequential changes in the pages tested.
• To split out tests of individual features you need some kind of CallScript capability that lets you organize your tests easily into groups of related functionality.

But you are dealing with AJAX -- and it seems that you left out the most important aspect of all: test synchronization. Tests of AJAX type applications scream out for playback synchronization -- but not by adding Waits or Delays or Sleeps that never work in practice. There has to be a way built into your playback engine to interact with the DOM of the page and sense when it is OK to continue playback.

We believe if you have the right facilities in the test engine the functional testing job for an AJAX application can produce very good results.

We hope that will get a good discussion going. And, meanwhile, we invite AJAX application testers to send us an example AJAX application that they'd like us to test with eValid.

New Google Gmail Example Illustrates DOM Commands

We've heard from many sources that Google's Gmail application is a very difficult to test reliably. This recently appearing newsgroup posting is one example.

Yes, we agree. The latest Gmail version certainly is a tricky one. And, dropping your testing into the older, pure-HTML version as a way out is certainly not a real solution.

The Google Gmail application is very JavaScript intensive, includes some AJAX type synchronizations, and generally is very sophisticated in its use of both HTML and JavaScript (and AJAX) features.

So we figured, if eValid could do the Gmail application well then that would bode well for eValid use on other, similarly complex web applications.

Our sample Gmail test was to sign on to an existing Gmail account, compose an email message, send it to the same Gmail account, observe that it arrives, then delete the message just recieved. This test sequence is "idempotent" (i.e. cleans up after doing its work and returns everything to the original state), it exercises the main Gmail functions, and it allows us to illustrate a variety of uses of eValid's DOM Element Manipulation/Motion commands.

We put this script through some very stressful playback checkouts. One test was to set up a 100-run sequence using a delay multiplier of 0.1 and that produced normal results. (Lower delay multipliers on a fast driver machine actually played ahead of the Gmail system response time.)

Here is the complete explanation of the Testing Google Gmail With Index/Motion Commands example. There's an important cautionary caveat: In this script if the round-trip test message is sent to the Trash area rather than to the InBox area where it is expected, eValid will -- quite correctly -- see this as an error, a true regression.

Use of eVgen Feature to Test A Combinatoric Application

A possible solution to testing a web application with apparent high-count combinations of input may be to use a continual testing process with randomly chosen values for each test case.

For a very large permutation count you will never try every combination. But over time, as the total number of runs increases, and the total number of combinations tried increases, you will be able to accumulate sufficiently many successful runs to generate some confidence in the integrity of your application.

Here is how this could work in practice.

• Create your test starting with an eValid recording of your web application behavior.
• Parameterize that script with variables whose values are to be assigned at the start of each test.
• Construct a table of possible values for each parameter.
• Use eVgenRand (an eValid suite component) to create the values for each instance of playback using the "random selection" option, and specify a large number of tries to make.
• Make sure errors are recorded somehow for later study.

If tests each take one minute, after a week you have about 10,000 tries and some number, N, of tests that fail. Then:

• If N continues to be zero, then you can increased confidence in the quality of you web application. The more you run the better you feel.
• For each of the N failures you have hard evidence of a problem that needs to be fixed.

This may not be as efficient as proposed schemes from the R&D community that attempt to reduce combinatoric complexity from the outset. But this solution has the advantage of easy implementation, and immediate availability. The relevant eValid commands involved are described in the eVgen Example: Yahoo! Random Search example.

Michael Hunter Interview In Dr. Dobb's Portal

There is an interesting interview by Michael J. Hunter with Edward Miller that has just appeared Dr. Dobb' Portal's Testing and Debugging section. The interview touches on some aspects of the origin of testing technology, on various difficult (and infamous) software quality problems, and what the challenges are for the future of software testing in a web-unified world.

The original copy of this interview that appeared in Dr. Dobbs (www.drdobbs.com) appears to have disappared.

Here is a link to the source of the original article:

    http://www.e-valid.com/Promotion/TradePress/Hunter.blog.html

Welcome to New eValid Users

We'd like to welcome the following new and continuing eValid customers to the growing eValid user community:

• Aricent Technology (India)
• B+S Card Services (Germany)
• Biznode Informatics (Sweden)
• COBB EMC (USA)
• DataPath (USA)
• Design Within Reach (USA)
• RegOnLine (USA)
• Symcor, Inc. (Canada)
• William Sonoma (USA)

Here's an expanded list of several hundred Selected Customers by Industry Sector.

New DOM Analysis Command Supports Regular Epression DOM Searching

A new eValid command, IndexFindElementEx has been added to the set of DOM manipulation commands. This new command searches the current DOM for a regular-expression match on multiple properties. This turns out to be a kind of "ultimate DOM search" command in the sense that — as far as we can tell — there is practically nothing that can't be searched for on a page.

This command is part of the set of DOM Element Motion/Manipulation commands that are used to construct position/structure independent scripts that operate entirely at the DOM level. Such scripts are particularly attractive because of their ability to tolerate a very wide range of page variability without losing their meaning and effect.