eVlite Benchmarks Updated

There are new benchmark data available for eValid's eVlite Limited Fidelity Playback feature. At the high end of capacity, our new benchmarks show that, using a fairly fast PC that is running up to 20 copies of eVlite at one time, you can simulate the activity of 20,000 user playbacks simultaneously.

Remember, eVlite activity is "non-coherent" in the sense that there is no session memory or context involved. eVlite playbacks are similar to those normally available when using a loading engine based on HTTP protocol activity. In the eValid context, eVlite work is used to create a background of high activity while measuring the performance of realistic, in the foreground, context sensitive test sessions.

URL Trace Generation Feature

The new URL Trace Creation feature is intended to make it possible to run tests, for comparison purposes, in eValid that mimic some types of less-sophisticated browser traffic generators such as those based in HTTP sequences.

In operation the new URL Trace Creation feature generates a derived script at playback time that contains the specific sequence of GetURL commands to account for every URL that the eValid browser downloaded during the script playback. The resulting derived eValid script can be used in different ways: for server load timing experiments, to create non-browser server activity, and for performance comparison purposes.

Total URL-only playback time typically exceeds that of eValid browser based playback because the browser activity is multi-threaded, whereas the sequence of GetURL commands executes serially (single thread, no parallelism).

SJSU Course on Software Test Automation Features evalid

Students in Prof. Jerry Gao's course on Software Test Automation (CMPE 296X) scheduled for the Spring 2008 will have access to eValid as part of the regular coursework.

GWT Conference: 3-6 December, San Francisco

Addison-Wesley and Prentice Hall are sponsoring on a conference dedicated to GWT: Voices That Matter: Google Web Toolkit has major support from Google staffers (e.g. Bruce Johnson, Joel Webber, Josh Bloch) who are responsible for the GTW system for creating Rich Internet Applications (RIAs) efficiently.

What is most important about this conference is that it focuses a great deal of attention to the use of Java + GWT as a powerful and efficient way to enhance the client-side user experience. It's the productivity that you gain this way that's most significant!

New Monitoring Support Commands Introduced

The most recent eValid release includes two new commands aimed at making the job of timing complete sessions easier and more logical:

• The TimeStamp command lets you create a separate file that records the exact time -- in elapsed playback seconds -- for various checkpoints in your script.
  
• The SaveReport command creates a separate file, with messages under user control, depending on where the commands are found in the playback script.

The main advantage of these commands is that you can run a multi-step monitoring playback, but break up the time phases within the playback session into meaningful sections, e.g. getting ready, loggin on, selecting something, moving to checkout, choosing the shipping, etc.

New eVlite Benchmarks Completed

There are new benchmark data available for eValid's eVlite Limited Fidelity Playback feature.

At the high end of capacity, our new benchmarks show that, using a fairly fast PC that is running up to 20 copies of eVlite at one time, you can simulate the activity of 20,000 user playbacks simultaneously.

Remember, this activity is "non-coherent" in the sense that there is no session memory or context involved. eVlite playbacks are similar to those normally available when using a loading engine based on HTTP protocol activity. In the eValid context, eVlite work is used to create a background of high activity while measuring the performance of realistic, in the foreground, context sensitive test sessions.

Index Commands Augmented

Two new commands, IndexSubmitClick and IndexInputValue, have been added to the group of commands that allow for manipulation of DOM index values.

These commands provide the eValid user with the ability to identify an object on the face of a page and to either click on it or feed it a value -- independent of where the object is on the page.

STAN, PROF, PAGEMAP Feature Normalization

It's housekeeping time! Over the last several years quite a few new eValid commands have been introduced, but in some cases we have not been as consistent as we should have been. Some commands' siblings -- with nearly identical functionality -- require different levels of playback feature key. It was time to normalize, and we have done that.

Some commands that were PROF level now are STAN, and some that were PROF now require PAGEMAP. If you are advised, after you update to the latest eValid version, that some commands now require a higher level feature key that you have, let us know and we will make the appropriate modifications to your license file.

Validation of Document URL Command Extended

The new Document URL Validation Extension feature provides a method for checking page URLs in more detail than before. In addition to validating the complete URL (the original capability) you now can validate the server name, the protocol type, or even a partial URL.

New Site Analysis Report Structure

We have revised the organization and presentation of the site analysis reports -- nearly twenty of them -- to make them easier to use and easier to manipulate. Here is the description of the new Site Analysis Report Selection page.

eValid site analysis users will also appreciate a rework of the material that describes the setup of the eValid spidering process by using the eValid Site Analysis Settings page.

During the spidering process, eValid applies "filters" to each page that is downloaded, and these are set up as described in the eValid Filters Setup page.

Google Search and Visible Text Result Confirmation

There was a presentation at the the GTAC conference aimed at finding ways to confirm correct and well-performing operation of multiple Google search engine top pages from a variety of countries. This example was prompted by that presentation.

The test requirement is to confirm, in a very simple manner if possible, whether a standard search page (with standard namings of elements), used in a variety of countries, works acceptably well. And, maybe along the way, figure out how long a chosen search actually takes.

The eValid solution uses the built in "motion commands" so that the resulting script that does the work parametrically is very short and is 100% desktop safe.

Full details are found at Google Search and Visible Text Confirmation. You'll see that we were able to run 15 instances of the script -- i.e. for 15 different Google home pages -- in just under 40 seconds. Comments on how to further simplify this are welcome!

Rich Internet Application Monitoring Support

Complex web-based applications, increasingly implemented with any of the many available technologies for "rich internet applications (RIA)" interaction, are often business-critical applications: You REALLY want to know they are working well!

eValid functional test playbacks, done on a regular basis, with the correct error flag processing options, are an ideal way to monitor RIAs. What's even better, you can have 100% confidence that you can get reliable playbacks -- we have a 100% Guarantee that you can get reliable playback.

eValid supports monitoring mode operations in a variety of ways:

• Do-It-Yourself Monitoring
  You can run eValid playbacks on your network locally, using eValid playback scripts can confirm performance thresholds, validate content at many levels of detail, time page downloads, and report with email and built-in charting. All you need are some copies of the eValid Application Monitoring Bundle.
  
  

• Agent-Based Monitoring System Reporting
  eValid scripts can do the actual RIA tests, and export the results to your local network monitoring environment.
  

  Nagios is one of the most popular of the network monitoring systems, and we have Nagios support available off the shelf. Plus, we have integrated eValid with a variety of other network management/monitoring solutions.
  




• Hosted Monitoring
  
  We can run your RIA tests -- scripts that you have developed or scripts that we create based on your specification -- in our own network. You get email alerts and notifications, and you have access to the long-term history with your own password-protected account. Here is our Monitoring Services Summary.

Let us know how we can help you get high quality RIA monitoring going for you -- so you can have confidence in your web application's public facing performance and behavior.

Google Test Automation Conference

Google's leadership in the search engine community makes it clear that they are "doing something right". And this extends to the technology area as well.

The upcoming Google Test Automation Conference (GTAC), 23-24 August 2007 in New York is their second foray into the software testing technology area. The link gives the speakers and their topics. GTAC's an invitation only event, but Google promises that videos of all of the presentations will be made public after the conference is over.

And, if you are concerned about test automation technology you may want to keep an eye on Google's Testing Blog.

A Newsgroup Question about eValid, Watir, and Selenium

Recently one of the newsgroups that we monitor regularly posted the following question:

I can't understand why anyone would buy a web-based test-tool when there are solid open-source alternatives, like Watir and Selenium. How would you differentiate your closed-source tool to these?

eValid's Response
eValid is a fundamentally different architecture from Watir and Selenium and therefore offers much different capabilities.

• The Watir Web Application Testing in Ruby application drives an IE browser from Ruby scripting, an approach that imposes a range of limitations, (which the Watir site admits): no support for Applets, Flash, and ActiveX components; no support for JScript generated popups; and no recording capability, etc.
  
• Selenium uses JScript and Iframes to embed playback action in a browser, an approach that precludes browser-internal object interactions and prevents any "direct to the browser face" interactions. Applets, Flash, and ActiveX can't be handled. There are additional access, security, and synchronization limitations, as described in the Against Javascript writeup.

• eValid is an IE-equivalent browser built using components and libraries from the Windows software development environment that provide direct low-overhead access to every browser capability.



Because eValid *IS* a browser it has the capability to handle crucial activities such as: opaque object (Applets, Flash, ActiveX) support; playback synchronization; adaptive playback; dynamic DOM validation; event synchronization; activity timing; application-mode operation; etc. All of these activities are integrated seamlessly in a GUI-driven system that records test scripts, plays them back, and analyzes, validates, and times test events.

• eValid use does not involve step-by-step "training" during script creation, script playback (in single or multiple-browser mode), or basic results analysis. The batch mode interface lets eValid act as the engine in regression test activities and server loading or as a Rich Internet Application (RIA) monitoring agent that reliably runs 1,000's of playbacks per hour.

• eValid is a commercial product that is fully supported, has maintenance subscriptions, and has a significant installed base of users. In continuous development for 5+ years (the current release is Version 8), eValid is a mature and stable product. The ongoing development activity ensures that eValid can meet the challenge of testing and analyzing new kinds of web-browser enabled applications.
  

Web Quality Workshop Includes ISTI Project Description

The upcoming Workshop on Web Quality, Verification and Validation being held in beautiful Como, Italy, 16-20 July 2007 includes a paper and presentation by I. Beicoglio, M. Fusani, G. Lami and G. Trentanni (all from the Institute of Science and Technologies in Informatics (ISTI), part of the CNR complex located in Pisa, Italy) that describes their project in website quality analysis.

Their paper, Establishing a Quality-Model Based Evaluation Process for Web Sites (PDF), describes how they will be working to establish a quality model based on eValid data collection and analysis. Ultimately their ISTI work will benefit public and commercial entities throughout Italy.

These workshop where their work is being presented is part of the International Conference on Web Engineering 2007 that was mentioned earlier here.