New Year's Greeting

As we come to the end of the calendar year it seems to be shaping up that 2008 will have been a really tough year for a lot of people. So we're doubly thankful that for eValid it has been a fairly good year.

Recall that the Chinese word for "crisis" is composed of two symbols: one for "danger" and the other for "opportunity."

Crisis = "dangerous opportunity" is the way that's usually interpreted. And, we aim to take advantage of the opportunities that the current situation presents!

So this may be a good opportunity for those of us who are in the quality and productivity business. It seems to us that products and technologies that make life easier, get the work done quicker and more efficiently, make better use of resources, and do the job better are going to be extremely important as the world economy gets back on track.

Here is a summary of what you can expect from eValid in the coming days and weeks:

• V9 Release, sometime after the first of the year with many new features and performance improvements. (All V8 support will be ending a few months after V9 is released.)
  
  

• There'll be several special offers for regression suite creation, for package pricing involving server loading work, and for monitoring script development.
  
  

• Realigned product bundle compositions that will see the EPI feature added to both the WebMaster Bundle and to the Regression Test Bundle. Look for other changes in product composition and organization, too.
  
  

• Revisions of some of the pricing, including introduction of SAAS-type pricing and special virtualization support.
  
  

• Enhanced DOM programming support: you'll now be able to do ANYthing you can imagine doing with DOM scripting based on "from life" recording.
  

We wish you all the very happiest of New Years celebrations, and we wish for the very best for everyone in 2009.

Simplified Cost Per User Calculation

Users of the eValid functional testing and server loading solution often ask about the actual costs over time "per simulated user". (Remember, eValid has no "virtual users" -- only simulated actual users.) So we made up a little table that reflects a typical eValid installation.

Machine Configuration -- Number of Master SLBs	1
Machine Configuration -- Number of Slave SLBs (Playback Only)	9
Minimum simulated user capacity per machine (XP)	125
Total simulated user capacity	1,000
Normal business hours in a year (40 hours/week, 52 weeks).	2,080
Total simulated user hours available in a year.	2,080,000
Average cost per simulated user hour based on a $10K investment in eValid.	<0.5¢

The point is this: eValid offers a potential average cost of < 0.5¢ per simulated user hour when spread out over an entire year. If you spread out the work to the full 168 hours per week the costs per simulated user hour drops even more!

About JavaScript Regressions

We've had a number of questions recently about problems that arise from changes that happen at the JavaScript level. The main point that users find is that a page that PASSes all their tests without difficulty suddenly beings to FAIL the test. Yet a visual inspection of the page shows no differences!

We know of an increasing number of applications which have different JavaScript depending on which browser version you're using (e.g. IE 6, or IE 7, or IE 8), or which operating system you're using (e.g. XP, Vista, NT2000, W2003S). The underlying server delivers different JavaScript based on what it knows about the client, and if your tests are engineered for one combination they may need some modification to work on another combination.

Here's our detailed explanation about how to handle JavaScript Regression.

Conferences of Interest

Three conference announcements that are worthy of considering are:

• TAP: Tests And Proofs 2009: Maybe testing and proving -- which have traditionally had diverging goals and methods -- will be able to merge?
• ISSTA: A long running conference that promises even more-focused technology in the upcoming rendition of this event.
• 7th Workshop on Software Quality: A workshop on a wide range of software quality issues, intended to be part of widely-respected ICSE 2009.

IT/QA Solutions for Tough Times

Nobody will argue with the sentiment that times are very tough. But times have been tough before, and somehow we got through them. And then things got better.

In times of budget cuts, hiring freezes and diminished headcount, it makes sense to pay special attention to the "total cost of ownership" for technical quality assurance, monitoring and performance validation solutions, because nobody can afford the costs of lost customers.

If you figure the benefits of gained productivity, ease of use, quickness to good results, overall flexibility, lower infrastructure and labor costs, we think eValid is truly a "tough time solution," where you can get a lot done with little effort, and at a very moderate price. Applying/introducing eValid in your IT/QA department will help cut costs and reap big dividends.

It's something to think about, so let us hear how eValid can be the cost effective alternative that helps your organization remain competitive and be positioned for success.

A Simple DOM Manipulation Example

We have been getting a lot of questions recently about how eValid handles interactions that require pure-DOM operations.

Of course, most of what you can do by constructing an eValid script to manipulate the DOM can be done with a recorded (captured) script, but in some cases there are conveninence and generality factors involved.

For example, you might want the scrip to be used in a systematic over very large numbers of differenc instances. Or, you may want to confirm/validate non-obvious internal details about DOM element settings?

Here's our simple illustration script that uses the main DOM manipulations Illustration of DOM Manipulation Capabilities. This example uses most of the capabilites: value input/output, the ability to find particular DOM elements, and the ability to act on them by modify values DOM attibute level.

Mutation Is Back

Sometimes good ideas never make it, and sometimes they do. Mutation analysis, a notion that goes back to the 1980's (and generated a big controversy in the test community then), appears headed for renewed interest.

Check out the 4th International Workshop on Mutation Analysis, part of the ICST 2009 Conference set for April 2009 in Boulder, Colorado, USA.

For those who don't know what the fuss is about, mutation analysis methods test the quality of a test suite by applying it to versions of the software product under test that have had "mutations" seeded inside; if the suite detects the errors, great; if it doesn't, then you know something is deficient. By the way, "product" here doesn't have to be live code (although it could be) and may include specifications, designs, or other types of objects It's an idea that may be worth looking into.

Welcome to New eValid Users

We're very pleased to welcome the following new eValid users:

Aricent (India) Bisnode Information (Sweden) Defense Contract Management Center Dice.com

Electrabel (Belgium) Loyola University Momentum Advanced Solutions Peterson Holding RegOnline Remindo

University of Southern California University of Pittsburgh Medical Center U.S. Department of Labor WebMD

We appreciate your business!

Using eValid To Study Page Dynamics

As web applications grow in complexity it sometimes becomes increasingly difficult to see what is really going on as the browser interacts with the server. This is particularly true with AJAX applications because sometimes what you see on the page is, to put it simply, not quite what is really there.

This can be particularly challenging when there is near-continuous interaction between the application (running JScript in the browser) and the server, which may be continually updating an iframe on the current page.

Here is a "trick" in use of eValid that will help users see what is going on behind the scenes. You set Detailed Timings ON, and pause eValid in playback, watch the event log for the browser/server communication, and then use your application normally. Here's the detail (including a sample script): Observing Timing and Asynchronous Operations In Web Pages.

eValid Adopted For Loyola University Computer Science Course

eValid has been chosen for use in Loyola University's COMP 370 (Software Quality, Metrics and Testing), being taught by Adjunct Professor Plamen Petrov.

According to Prof. Petrov, the course's graduate and undergraduate students will use eValid to develop a detailed functional test suite for a web based application, and then critique the application in terms of quality, performance, and reliability using a range of eValid features.

Performance of IndexFindElementEx Command

We have done some experiments with the IndexFindElementEx command that compare its performance with the simpler IndexFindElement command. We were looking into this because we noted that some Regular Expression searches seemed to be taking a rather long time.

The performance difference arises because of the difference in searching the DOM dynamically for a simple string match for an atribute, versus doing the same search for a Regular Expression (RegEX) match. The RegEx search turns out to be a great deal more work. Our results appear to indicate that, for a typical complex web page (think of www.cnn.com) the regular expression match process takes about 8-12 times as long as the simple string match. This tends to be the the result of having to do so very much more work.

For example, at typical CNN page has ~2000 DOM elements, so a search of every element for a simple string uses about 2000 "string match" calls (all done in local memory). These matches insist that the string match from the left hand side (from the beginning) of the named property. Because the regular expression search has to start of at each possible character in each DOM value you wind up with quite a large number of searches. Fortunately, the work is all done in memory so the performance is very good!

On the other hand, if you search for a simple string simple string (one without any regular expression characters) using the IndexFindElementEx command, the process needs to examine each DOM element once for each character in the string, to see if it might be a match. As you can guess, this can be a lot fewer searches, and the performance is very quick.

For example, one regular expression search we ran on CNN made 325,000 attempted matches within the DOM for a regular expression, whereas only about 6,100 were needed for a regular (simple) string match. That's ~50 times as much work to the regular expression matching, depending on the lengths of the strings involved. So it is reasonable that when a regular string match on CNN.com takes ~0.2 seconds, the full regular expression match will take ~10.0 seconds. Faster machines will show reduced search times. See DOM Analysis Performance Benchmarks.

Handling Security Issues in Vista

We've heard from some users that Vista's enhanced security is interfering with some eValid test playbacks, and with some eValid site analysis runs. We're certainly in favor of Vista's enhanced security, but for production purposes in case you wish to modify that here is the way to do it.

Here's a writeup on how to do this: How to Suppress Vista Security Pop-Up Windows.

Current Conferences of Interest

From time to time it's important to take note of goings on in the software testing community. Here are some recent conferences that may be of interest:

• International Conference on Software Testing and QA, Bilbao, Spain, 29-31 October 2008.
• Automatic Program Verification: APV 2009, Argentina, 14-15 February, 2009,
• Workshop on Testing Techniques and Experimentation, part of the ICST 2009, 1-4 April 2009, Denver, Colorado, USA.
• 19th International Symposium on Software Reliability Engineering, 11-14 November 2008, Richmond, Washington, USA.

Product Updates

Wouldn't you know it: just when we have everything running smoothly, something changes and, well, it doesn't run smoothly any more! The happened in the last couple of weeks when users of eValid on Vista experienced license problems. We traced the issue to changes due to the latest Vista SP, and now it's 100% resolved in the current eValid build.

Manwhile, we checked out the latest IE8 Beta2 release and found that all's well with eValid with that version of IE installed. Perhaps more important, it does appear that on some web sites eValid does behave noticeable faster that before when using the IE8 Beta 2 DLLs.

Finally, there have been some across-the board product performance enhancements and some new capabilities.

• ExitNow has been reworked so it really is definitive: eValid hard-exits on this command.
• The option to encode password details (to keep out prying eyes) has been extended to the AuthenticateUser command.
• Changes were made to several of the "startup" dialogs to improve readability.
• The maximum length of the name/value pair (query string) section of URLs has been modified to account for arbitrary length URLs (very long strings were failing randomly).

Automated Verification Technology Conferences

Announcements for these two events arrived almost simultaneously the other day, and maybe that's significant? Two events focused on automated technology for verification of software systems.

The ATVA 2008 Conference is set for Seoul, Korea, 20-23 October 2008, and aims to promote research and dvelopment on theoretical and practical applications of automated analysis methods...with an emphasis on applications in East Asia.

The AVOCS'08 Workshop is set for 30 September to 01 October 2008, Glasgow, Scotland, and also focuses on trying to find out how to get automated analysis and verification systems into practical use, with an emphasis on applications in the European theatre.

Both worth checking out.

About Warnings Vs. Errors

Many eValid users don't understand what Warning messages mean. Here's a quick rundown:

• If you are using the Playback Data Dialog, you'll only see a Green Screen when the playback runs to completion without issuing any playback flags at all, i.e. no Warning, Error, Timeout, Alarm, or Failure flags.
• You'll see a Red screen on any Error, Timeout, Alarm or Failure flags.
• You'll see a Yellow screen when there are any Warning flags. The most common reason for a Yellow screen is that the Adaptive Playback system had to adjust the playback behavior to overcome minor changes (regressions) in the web page.

Here's a full explanation of eValid's Error Reporting Process.

EPI-Based Program Implements Gmail Example

As an illustration of the power of the eValid EPI (eValid Programatic Interface) we've worked up the CPP programs for the Gmail Example that itself shows of the capability of eValid's DOM Access Capability.

The main things to appreciate in this example are that (1) the pure-playback eValid script does not require direct access to the desktop, all of the activity being performed at the DOM level, and (2) the corresponding EPI-based CPP program runs the same identical script as a windows direct executable.

Conversation With a Test Manager (Part III)

In Part I and Part II we related how eValid dealt with concerns about eValid technology and about reliance on the record/play paragigm. Here we finish up looking at issues of operational complexity and overall field costs.

• "Effective Use of eValid Will Be Obscured By The Complexity Of The Scripting..."
  Actually, it's the opposite. eValid's approach is to AVOID scripting as much as possible and to resort to manual script modifications only as a last resort. It's cheaper that way.

  We have found that 99.99% of the time the "from life" recorded script is adequate as it stands, mainly because Adaptive Playback (defaulted ON) softens the "brittle nature" of the script/application correspondence.

  In practice, users record scripts from life and then modify and/or augment the generated script by manually adding commands (e.g. AJAX synchronizations) as required. You don't have to remember any script syntax -- eValid does that for you. This process is enormously more efficient that hand-coding tests from first principles -- which eValid users normally never need to do.

  The Testing the GWT "Kitchen Sink" example scripts show some of these modifications, e.g. when extra validation steps were added to confirm that the playback arrived at the correct end state.

  An even better example of this is a sample test of Gmail, which shows a number of hand-modified advanced-usages of AJAX synchronization command is our Testing Google Gmail with Index/Motion Commands.

• "But, Hey, That's Great But I'm Really Concerned About Productivity Costs..."
  eValid is first and foremost a productivity tool. For web applications eValid's automated script recording, augmented with a few hand modification of scripts based on some knowledge of the available "extra" commands, and in a context that includes AdaptivePlayback, easily trumps every other approach in terms of productivity.

  The famous testing notion "...if it ain't broke, you're not trying hard enough!" is only going to be effective if you can try things out in sufficient quantity AND quality. Think functional coverage here. A script that costs $5,000 to develop and that tests one feature brilliantly is never going to have the quality enhancement effect of 100 scripts that cost $50 each to develop, because these 100 tests may cover 100 different functions. The more different things you test the better off you are.

We hope you enjoyed our interview!

Corporate e-Communications Award (CECA)

eValid, our South Africa business partner BlueRiverStone, and a number of additional sponsorsing organizations, are mounting the Corporate e-Communication Award for 2008. Awards for this study are expected to be announced in mid-October 2008.

eValid's role in this competitive analysis is to perform the detailed technical evaluations of the candidate companies' websites. After we complete the scans we publish the information collected to each organization so they can have the opportunity to update their website to overcome any anomalies difficulties that the eValid scan identified.

In prior years a similar eValid methodology was used, to very good effect, for several similar website quality competitions and contests, some reported in the Financial Mail, and some published independently by BlueRiverStone. Here are several of the reports.

• South Africa Travel and Tourism Industry (OTTI) Award
• Who's Hot, Who's Not (BCW 2005)
• The Best, And The Rest (BCW 2004)
• Financial Times FT-150 Website Comparison, and also FM Top 100 Websites That Save You Time & Money

In all of these studies, the eValid site analysis scans were used as the basis of the the technical analysis scores, using data from runs we made from our facility here in San Francisco. Kudos to Ian Kruger for his great work in exploiting the data derived from this super eValid application!

We'll bring you more details about how the contest works out in future entires.

Conversation With a Test Manager (Part II)

In Part I of our conversation with the high-level big-company test manager we looked at how eValid technology works in some detail. Our conversation then turned to criticism of our use of the "record/play" paradigm. He believed that record/play is a throw-back to an earlier era, and doesn't apply to web based applications.

• But eValid's only a Record/Playback Tool, and Everyone Knows That's Passe..
  Actually, the record/playback method is actually very powerful if two conditions are met: (i) you need a strong recording engine and (ii) you need a strong playback engine. Without both we kind of agree with you -- you might be wasting your time.

  We were able to achieve both goals because all of the record/playback functionality is built inside the browser, where there is "perfect" information about the user's actions, about the current page, and about state changes an action causes.

  • Recording: The recording engine (the technically most difficult part of eValid) operates with full browser state knowledge. The result is a compact and reliable test recorder that creates scripts that play back very accurately. Generated commands are shown in the script window during recording virtually simultaneously with user actions. It's easy to see what is working and when Alternative Recording Modes need to be used.
  • Playback: Brittle tests are indeed problematic. In our view, eValid's Adaptive Playback feature overcomes that by taking advantage of a playback engine that is fully informed about all the details in the current page. It can do the "right thing" and prevent loss of valuable script data.
• "eValid Users Will Require Intimate Knowledge of How User Inputs Are Interpreted..."
  The eValid recording engine senses user actions in the DOM context. It uses browser-internal access to these signals and events, watching for them to happen (recording) and/or causing them to happen (playback). The automatic recording/scripting often fills in for needing detailed application knowledge.

  It is worth noting that the rather longish scripts in our Testing the GWT "Kitchen Sink" example were created 100% by recording -- they certainly were not scripted by hand (God forbid!). The longest one, if you check the recording start/end times, required about 90 seconds to actually record from life -- and that includes the recorded think times!

  The user behaves normally, and eValid automatically acquires the intimate knowledge of how user inputs are interpreted, and writes the commands accordingly. Very little training on how to do this is ever involved. (AJAX applications may be an exception, but you would expect that.)

• "But What About Actions That Are Interpreted by On-board JavaScript..."
  eValid is an IE clone; it implements the full IE JavaScript engine. JavaScript events are sensed internally during recording, and JavaScript events are triggered directly in the eValid browser to accomplish playback effects.

  It is VERY important to note that -- unlike systems like Selenium and several others -- eValid playback is NOT done with JavaScript. Script playback in eValid is via direct interaction with the browser -- inside the browser.

  This architecture has some neat benefits: eValid never encounters problems with the "same-origin policy", security zones and authentications, the "can't write to a file" constraint, cross-site scripting limits, or any of the other limits intrinsic to JavaScript-based test playback solutions.

  These are just a few of the intrinsic limits of JavaScript based playback solutions. The eValid advantage, in NOT being a JavaScript based solution but instead being based on use of a full-featured browser, becomes even more apparent when you need to synchronize an AJAX application's test playback. eValid does AJAX synchronizations natively, by repeatedly interrogating the DOM internally to wait for a DOM element attribute to take on a required value, withOUT interfering with the concurrent execution of JavaScript that supports the AJAX application.

  Because of these limitations we believe JavaScript-based playback methods are a technical dead end.

That covers the main points about eValid's architecture. In Part III we'll see how eValid shapes up as a practical system for delivering tests.

About Warnings Vs. Errors

Many eValid users don't understand what Warning messages mean. Here's a quick rundown:

• If you are using the Playback Data Dialog, you'll only see a Green Screen when the playback runs to completion without issuing any playback flags at all, i.e. no Warning, Error, Timeout, Alarm, or Failure flags.
• You'll see a Red screen on any Error, Timeout, Alarm or Failure flags.
• You'll see a Yellow screen when there are any Warning flags. The most common reason for a Yellow screen is that the Adaptive Playback system had to adjust the playback behavior to overcome minor changes (regressions) in the web page.

Here's a full explanation of eValid's Error Reporting Process. The key notion here is this: Warnings aren't Errors. Unless they are.

Conversation With a Test Manager (Part I: The Basics)

A while ago we had a long conversation with a senior test manager at a big company about eValid, about eValid's technology, and about how this might all play out in his environment of tester/programmers. It was quite a long conversation, and we didn't make a recording (only notes), but here are some of the main issues he raised and how eValid answers them.

But eValid Only Runs On Windows, And Only On IE...
Yes, that's true right now, but eValid technology is currently being implemented in the FireFox build tree, using the current product as the design base. In the meantime, eValid already can "set user agent string" so that it can pretend to be ANY browser. We even have a fully worked
Illustration of User Agent String Modification.

But, Hey, eValid's Scripting Language is Non-Standard...
Actually, the eValid script language is more an automatically generated "command language" than it is a pure scripting (programming) language. We worked up a view of the eValid script language as a abstraction that implements a user controlled browser. The Abstract Model of eValid: Script Language View is the result. All of the eValid commands easily map into simple function calls (method invocations), following the internal Script Principles description.

But We Are Used to Writing Very Complex Testing Programs...
For applications where it is required, the EPI (eValid Programmatic Interface) provides users with full programming language support. The EPI feature is currently available for C++; versions for Java, VB, and PERL are being implemented.

Do You Expect Us To Guess the Properties of Page Elements...
Not at all. Access to DOM objects and all their detailed properties is done with the PageMap feature. Information from the PageMap display is bi-directional: you can click on the tree and/or click on the page, and you see the DOM specifics or page elements immediately.

So much for eValid technology and some of the main features. In Part II we'll look at the issues surrounding eValid's use of the "record/play" paradigm.

Control Mode Transition Diagram

During a test recording eValid can be in a number of states, and the current state affects how user actions are recorded in the script. The Control Mode Diagram indicates how these modes interact. Here is a quick summary of the available control mode toggles:

* F11: Absolute clicks

* Ctrl-F11: Absolute mouseovers

* Alt-F11: Element mouseovers

* F12: Application Mode

* Ctrl-F12: Desktop Mode

In practice when you are making a recording you may need to toggle into and out of these modes based on what you see eValid recording in the script window. It's a good bet that if you take an action and nothing happens in the script window that you will either (a) need to use one of the above-defined options or (b) mark that part of your script as needing manual modification.

One way or the other, you can always arrive at a script that plays back reliably. 99.99% of the time that's just by recording "from life".

EPI C++ Example Programs

We have developed some simple examples that show off how eValid's Programmatic Interface (EPI) works.

Example #1: This very short example involves just a few evalid commands, an InitLink and a GotoLink. It's main purpose is to illustrate how the EPI interface is structured from the programmers' point of view. You can see both the CPP program and the corresponding non-EPI eValid script for comparison.

Example #2: Similar to the above, this example also is 1:1 with a corresponding eValid script. This script involves running an IndexFind command that identifies [to an internal variable] a particular field. The same command in the EPI interface performs the same way but in this case the actual index value is returned into the CPP program. That index is available to be used for other purposes in the CPP code.

Example #3: Here is a CPP program that performs a particular kind of search that can only be done easily with regular eValid command sequences of some complexity. The program uses a search engine to search for a particular search term, and then computes [by iterating and counting] the first search-results page that includes both the searched-for string and a target URL.

Example #4: Here there is a regular recorded eValid script that does a number of different kinds of things, and the EPI CPP program which duplicates the script actions exactly.

Example #5: The program in this example does not have a direct eValid script equivalent. The program navigates to a user-specified URL and then, within that page, finds the specific index of the [first] element in that page for which a specified attribute NAME has a specified VALUE. This read is done from the DOM of the page as it currently exists, which may or may not be what was sent down from the server, depending on what kind of AJAX-type operations were performed on it.

Example #6: The program in this example does not have a direct eValid script equivalent. Similar to Example #5, this program changes the value of attribute NAME at a specified index on the page to the specified VALUE. This update action is done to the DOM of the page as it currently exists, without permission of the browser or the server. This is a powerful feature and user caution is advised.

The bottom line for this set of examples is this: EPI for C++/CPP is available today and lets you do tests in an efficient and powerful way.

User Agent String Modification

We get a lot of questions about where eValid stands in relation to implementation on non-IE browsers, and in particular, when we'll have a Firefox version available. While we're not quite ready to announce any dates on that, you can rest assured that the Firefox version is on the product roadmap!

In the meantime, what can eValid offer to someone who wants to see if a particular application works (or perhaps equally as important, doesn't work) on a particular browser. It turns out that eValid has this nifty "SetUserAgent" command that may be helpful in that kind of testing.

The SetUserAgent command lets eValid "pretend" to be any browser you wish it to be, during a particular test session. This feature can be used to validate server behavior that involves alternative responses based on browser type (including, as the example will illustrate, denial of support to specific browsers). The User Agent string manipulation feature is also available at the command line interface.

Here is the Illustration of User Agent String Modification example. Let us know how you like it?

10th IEEE International Symposium on Web Site Evolution

This IEEE symposium WSE 2008, 3-4 October 2008, Beijing, China. (WSE 2008 is co-located with the 24th IEEE International Conference on Software Maintenance.) The paper and presentation schedule appears to be very strong, including special presentations by Scott Tilley ("Ten Years of Web Site Evolution") and two papers (one in each conference) by Harry Sneed, the noted German software engineering authority.

Demonstration Yahoo User Interface (YUI) Application

You've probably heard of GWT. But have you heard of YUI?

YUI, the Yahoo User Interface, is a collection of very powerful JavaScript libraries and "widgets" that offer a web developer -- and a an AJAX application developer -- a very powerful collection of ready-to-use objects and interfaces. YUI is widely used, as evidenced by this Yahoo! User Interface Library: Sites Powerd by YUI page.

eValid works well with these sites because the YUI approach is pure-JavaScript all the way. We worked up an example YUI (Yahoo User Interface) Demonstration Script to illustrate how this works. The example handles making a reservation on the SouthWest Airlines website.

This example shows that eValid handles YUI-built applications and also illustrates use of eValid's IndexMotion commands and internal timer commands to create a script that could serve well in functional testing, regression test, and monitoring test roles.

Google Testing Blog Response

There was a post on the Google Testing Blog (Motto: If it ain't broke, you're not trying hard enough!) that seems to be pleading for two things we've already developed.

In response in part to the Google Blog's notion of "layer of abstraction" as a way to characterize a solution we pointedout that we had already prepared a simple description of eValid in terms of its command language. We are also working on additional "abstract models for other aspects of the eValid solution Here's the item: eValid As An Abstract Model: An Initial Analysis.

Meanwhile, the Google Blog item also alluded to issues surrounding testing Google Web Toolkit applications. Testing GWT is straightforward for eValid, as our example showing tests of the GWT Kitchen Sink widgets illustrates. Here's the link: eValid Operation with GWT "Kitchen Sink"Confirmed.

Summary of AJAX Synchronization Support

Experienced web testers know this too well: AJAX applications are often very difficult to test reliably because they are, after all, inherently asynchronous (AJAX = Asynchronous JavaScript and XML). The problem is that a test recording, one made for example with the "real time recording" setting turned on, will play back perfectly well for a while, but will get out of synchronization when the target website is running slower. The asynchronous JavaScript executions almost always fail to sync using simple Wait times. Something more -- much more -- is needed, and eValid has what is needed.

But not every test command (test action) requires extrinsic synchronization, and in many cases the extra step of having to figure out how to add in a synchronization step isn't even necessary because the intrinsic synchronization is good enough. This table summarizes all of eValid's Synchronization Modes. As you can see there are many different ways that playback synchronization can be maintained.

For tough AJAX applications you usually need to rely on one of the DOM-based synchronization modes, e.g. SyncOnElementProperty or SyncOnText. Those commands need to have their parameters filled in manually through use of the PageMap facility to find the right element properties (attributes) and their values. Our worked example that shows how we test the Google Gmail Example is a good illustration of how this works out in practice

3rd Google Test Automation Conference (GTAC)

Google sponsored the first GTAC in New York last August, and now they have organized a second rendition. The official Call For Attendance: GTAC 2008 gives details about how to apply for a spot to attend. Google is very generous: it pays for the conference in its entirety.

Here's the list of scheduled presentations (in alphabetic order by paper title):

• Advances in Automated Software Testing Technologies, by Elfriede Dustin
• Atom Publishing Protocol, Testing a Server Implementation, by David Calavera
• Automated Model-Based Testing of Web Applications, by Oluwaseun Akinmade and Prof. Atif M Memon
• Boosting Your Testing Productivity with Groovy, by Andres Almiray
• Deployment and Test Automation: Extending the Notion of 'Done' to 'System Tested on a Production Deployment', by Marc-Elian Begin
• JInjector: a Coverage and End-To-End Testing Framework for J2ME and RIM, by Julian Harty, Olivier Gaillard, and Michael Sama
• Practicing Testability in the Real World, by Vishal Chowdhary
• Taming the Beast: How to test an AJAX Application, by Markus Clermont and John Thomas
• The New Genomics: Software Development at Petabyte Scale, by Matt Wood
• The Value of Small Tests, by Christopher Semturs

Search Engine Optimization: Initial Analysis Script

A suggestion was made some months ago by one of our colleagues who is in the search engine optimization business that we ought to see if eValid could help her out. We asked her, "What is the most common question you want to have answered?"

After several go-arounds, her response was, basically, that it would be great to have an simple, fully-automated way to find out if a particular search string produced a valid clickable link in the first page of search results, and do this across multiple search engines. That second part of the request was the sticky one: just showing a text item was there on the results page was not enough. You really had to know there was a real live link that someone could click that would take you to a specific URL. Making one script for multiple search engines was going to be tricky, too.

The resulting SEO Demonstration Script is a parametric script, expressed in terms of the name of the search engine, the locations of the search-string entry field, the location and properties of the "go get it" action field, and of course the input search string and required URL fragment. To make the script 100% reliable -- we don't want to have to rewrite this script if the search engine pages change -- we worked out the Search Engine Parameters for a number of different search engines.

We ran the parametric script using one of eValid's methods for multiple-script playbacks (e.g. DataSynthesis mode among several options) with very interesting results. Natural enough we searched for "eValid" and discovered lots of references in places we didn't think we'd be referenced. Interesting indeed, the differences we found.

2nd IEEE International Conference on Software Testing, Verification and Validation

Here is a well intentioned, well-organized conference that might be of real interest to the web testing community: ICST 2009, 1-4 April 2009, Denver, Colorado, USA. One aspect of this conference worth noting is that the fact that it claims to have an "open management" approach (see the ICST Charter) -- so it may be a step away from the typical vendor-sponsored events that seem to be growing so rapidly in number.

Summary of Worked Examples

We've been developing already-worked examples of eValid tests of difficult websites or hard-to-test web applications for some time now, and there are quite a few of them completed. We thought it would be interesting to organize these in a way that shows off how example illustrates one or more features of the eValid test engine:

• Analysis of the DOM of the page for particular properties.
• Explicit playback synchronization based on page properties.
• Use of eValid page-position-independent "index motion" commands.
• Support for AJAX operations, including synchronization.
• Use of eValid's ability handle Regular-Expression matches.
• Use of one of eValid's many page-based validation modes.
  

The result is our Application Support Coverage Matrix, which describes what we have done so far.

New Movie Demonstrates Test Unification

There's a new eValid movie you might want to review that shows eValid operating in "test unification" mode. By "test unification" we mean the ability to have one test script server multiple purposes, e.g. for tuning properties of a page (how fast parts of it download), or confirming availabilty through monitoring, verifying non-regression (nothing's changed when nothing is supposed to have changed), or for inclusion in a multi-user server loading experiment.

The movie illustrates eValid's ability to do page tuning, functional testing, and server loading, all from the same base. The new 3WAY Movie [shows in a new window, duration 05:39] illustrates how simple this can be.

eValid As An Abstract Model: An Initial Analysis

A suggestion was made some months ago to work out how eValid "implements an abstraction". The idea of this is that any complete technology that addresses a class of problems solves it by lifting an abstraction from the problem class and solving that.

Doing that piece of abstraction analysis is a pretty tall order, and we are not finished with it. But we did recently complete a description of how eValid represents an abstract model that is the based on viewing eValid from its script language perspective.

As our Abstract Model of eValid: Script Language View points out, you can understand eValid quite well as a controllable browser upon which you can take particular actions to produce particular effects. We'll publish more of this set of descriptions as they become available.

How To Test AJAX Applications

There was an interesting post by Markus Clermont and John Thomas on the Google Testing Blog titled Taming the Beast (a.k.a. how to test AJAX applications) that got us to thinking about how much distance there is between what eValid can do and what the community needs -- for testing AJAX in particular. Here is the response we posted:

eValid's Response:
Interesting post, and yes we certainly agree that AJAX web applications can be quite difficult to test. But the picture you seem to paint is rather bleak, and it need not be.


The situation for testing an AJAX application is actually easier than you may think, provide that you have the right tools and by using their underlying technology base can "do the right things" at the right time.

• To overcome the expense of creating functional tests it makes sense to have a good test recorder, which takes away most of the pain of creating tests.
• Tests can be brittle, so you should have some kind of functional test playback feature than automatically adapts the bahavior of the playback engine to overcome inconsequential changes in the pages tested.
• To split out tests of individual features you need some kind of CallScript capability that lets you organize your tests easily into groups of related functionality.

But you are dealing with AJAX -- and it seems that you left out the most important aspect of all: test synchronization. Tests of AJAX type applications scream out for playback synchronization -- but not by adding Waits or Delays or Sleeps that never work in practice. There has to be a way built into your playback engine to interact with the DOM of the page and sense when it is OK to continue playback.

We believe if you have the right facilities in the test engine the functional testing job for an AJAX application can produce very good results.

We hope that will get a good discussion going. And, meanwhile, we invite AJAX application testers to send us an example AJAX application that they'd like us to test with eValid.

New Google Gmail Example Illustrates DOM Commands

We've heard from many sources that Google's Gmail application is a very difficult to test reliably. This recently appearing newsgroup posting is one example.

Yes, we agree. The latest Gmail version certainly is a tricky one. And, dropping your testing into the older, pure-HTML version as a way out is certainly not a real solution.

The Google Gmail application is very JavaScript intensive, includes some AJAX type synchronizations, and generally is very sophisticated in its use of both HTML and JavaScript (and AJAX) features.

So we figured, if eValid could do the Gmail application well then that would bode well for eValid use on other, similarly complex web applications.

Our sample Gmail test was to sign on to an existing Gmail account, compose an email message, send it to the same Gmail account, observe that it arrives, then delete the message just recieved. This test sequence is "idempotent" (i.e. cleans up after doing its work and returns everything to the original state), it exercises the main Gmail functions, and it allows us to illustrate a variety of uses of eValid's DOM Element Manipulation/Motion commands.

We put this script through some very stressful playback checkouts. One test was to set up a 100-run sequence using a delay multiplier of 0.1 and that produced normal results. (Lower delay multipliers on a fast driver machine actually played ahead of the Gmail system response time.)

Here is the complete explanation of the Testing Google Gmail With Index/Motion Commands example. There's an important cautionary caveat: In this script if the round-trip test message is sent to the Trash area rather than to the InBox area where it is expected, eValid will -- quite correctly -- see this as an error, a true regression.

Use of eVgen Feature to Test A Combinatoric Application

A possible solution to testing a web application with apparent high-count combinations of input may be to use a continual testing process with randomly chosen values for each test case.

For a very large permutation count you will never try every combination. But over time, as the total number of runs increases, and the total number of combinations tried increases, you will be able to accumulate sufficiently many successful runs to generate some confidence in the integrity of your application.

Here is how this could work in practice.

• Create your test starting with an eValid recording of your web application behavior.
• Parameterize that script with variables whose values are to be assigned at the start of each test.
• Construct a table of possible values for each parameter.
• Use eVgenRand (an eValid suite component) to create the values for each instance of playback using the "random selection" option, and specify a large number of tries to make.
• Make sure errors are recorded somehow for later study.

If tests each take one minute, after a week you have about 10,000 tries and some number, N, of tests that fail. Then:

• If N continues to be zero, then you can increased confidence in the quality of you web application. The more you run the better you feel.
• For each of the N failures you have hard evidence of a problem that needs to be fixed.

This may not be as efficient as proposed schemes from the R&D community that attempt to reduce combinatoric complexity from the outset. But this solution has the advantage of easy implementation, and immediate availability. The relevant eValid commands involved are described in the eVgen Example: Yahoo! Random Search example.

Michael Hunter Interview In Dr. Dobb's Portal

There is an interesting interview by Michael J. Hunter with Edward Miller that has just appeared Dr. Dobb' Portal's Testing and Debugging section. The interview touches on some aspects of the origin of testing technology, on various difficult (and infamous) software quality problems, and what the challenges are for the future of software testing in a web-unified world.

The original copy of this interview that appeared in Dr. Dobbs (www.drdobbs.com) appears to have disappared.

Here is a link to the source of the original article:

    http://www.e-valid.com/Promotion/TradePress/Hunter.blog.html

Welcome to New eValid Users

We'd like to welcome the following new and continuing eValid customers to the growing eValid user community:

• Aricent Technology (India)
• B+S Card Services (Germany)
• Biznode Informatics (Sweden)
• COBB EMC (USA)
• DataPath (USA)
• Design Within Reach (USA)
• RegOnLine (USA)
• Symcor, Inc. (Canada)
• William Sonoma (USA)

Here's an expanded list of several hundred Selected Customers by Industry Sector.

New DOM Analysis Command Supports Regular Epression DOM Searching

A new eValid command, IndexFindElementEx has been added to the set of DOM manipulation commands. This new command searches the current DOM for a regular-expression match on multiple properties. This turns out to be a kind of "ultimate DOM search" command in the sense that — as far as we can tell — there is practically nothing that can't be searched for on a page.

This command is part of the set of DOM Element Motion/Manipulation commands that are used to construct position/structure independent scripts that operate entirely at the DOM level. Such scripts are particularly attractive because of their ability to tolerate a very wide range of page variability without losing their meaning and effect.

AJAX Synchronized Testing Example

The AJAX Autocompletion Demo on Script.Aculo.us includes the major methods use in almost any AJAX application.

We've worked up a demo of how eValid reliably handles text input, DOM-based AJAX synchronization, input selection, and result validation activities on this example website. These are the same kinds of actions you would need to perform in testing any AJAX implementation. The eValid Example Script involves use of eValid DOM search, DOM synchronization, and DOM state identification commands.

New DOM Analysis Command Supports Event Firing

eValid users have been making use of the DOM-based "page motion" capability for some time. These commands let you identify an object in a page based on its properties, and then take an action on the page element you've identified. Scripts made this are particularly attractive because of their ability to tolerate a very wide range of page variability without the tests losing their meaning and effect.

A new eValid command, IndexElementEvent provides a capability in eValid to initiate any kind of action on a selected sourceIndex. The actions available this way are intended to include all possible user-initiated events. The new command is part of the DOM Element Motion/Manipulation commands that are used to construct position/structure independent scripts that operate entirely at the DOM level.

New Google Maps Example Illustrates Map Motion Validation

The Google Maps application is one of the oldest and most widely used AJAX applications. Building tests for it is a great way to exercise a web application test engine, so it is natural enough that eValid would use it to illustrate eValid capabilities.

One of the areas that eValid users ask about is the actual map area. How do you drag the display around and confirm (validate) that what you got to today is where you wanted to get to? This is the essence of a functional validation test for the application.

Moving the map display necessarily involves using desktop actions, so from the outset we know that the tests that involve this kind of work won't be "desktop safe". In spite of the desktop-required limitations, can these tests be made reliable and repeatable?

The answer turns out to be YES, as illustrated in this Google Maps Motion Consistency Demonstration Script. The main point about this example is that it performs two kinds of synchronization (one on the URL and one based on DOM values), and then reliably does the image drag operations, followed by a checksum verification of the resulting map image.

20th International Conference on Computer Aided Verification

CAV 2008.
The CAV conference has held a key role in software quality from a theoretical side as well as for practical applications. In most prior rendictions, and in this year's version as well, the CAV content has aimed at very high-level issues that directly or indirectly affect system reliability, fault tolerance, internal quality, and the like. The important -- and often unwisely underemphasized -- area of formal verification technology is unusually well-represented at CAV 2008.

13th ICRST, 16-20 June 2008, Venice, Italy

This the conference EVERYBODY wants to go to, because of the technical content of course, but perhaps also because of the location: ICRST 2008. You can tell that "location" is important when the top page of a conference web page opens up with a Google Map picture showing the location right on the banks of the Lido de Venezia. Think of the quality of the environment, and the food...

ICRST is one of the longest-running conferences in the field devoted to real-world applications of all kinds of quality technologies. Historically, ICRST's industrial presentations have been very stong, having become in effect the venue of choice for disclosure of important results on real-world reliabilty and quality work. This year's program follows that tradition.

Oh, one more detail that may or may not be important: it seems that one of the vaporetta stops is right in front of the conference venue. How convenient! How delightful!

Backbase/Duke's Bookstore Example

AJAX continues to be an area of strong interest in web testing, and in AJAX applications there seems to be two recurring issues: playback time access to the DOM, and playback synchronization.

To illustrate how to do both of these we've put together the Backbase/Duke's Bookstore Example & Demonstration Script. The main feature of the script is the use of a DOM value to achieve the required playback synchronization. To do this required some investigation of the page properties using the eValid PageMap feature. As you can see from the example script several variations of the SyncOnSelectedObjProperty command were used. Note that the synchronizations are done 100% inside the browser (i.e. they do not require use of the desktop); this means two or more of these same scripts could be running in parallel if you wished (or need to).

Testing the Google Gmail Application

The Google Gmail system is a widely used AJAX application that presents some special problems for testing, not the least of which is the process of synchronizing a test playback. We're all familiar with the "Loading..." message...and the problem in setting up a test script is how to make the playback wait until the mail file is ready to be processed

The solution, as described in our Testing the Google Gmail Application writeup, is to synchronize on a visible text entity. As you'll see in the example script, this means having the test playback wait until the string "You are currently using" appears on the screen. Note that this synchronization is done 100% inside the browser (i.e. it does not require use of the desktop); this means two or more of these same scripts could be running in parallel if you wished (or need to).

eValid Confirmed Operational on MS Windows XP/SP3

We're pleased to confirm that eValid V8 is fully checked-out on the most-recent version of Microsoft XP/SP3. This kind of confirmation is part part of our regular automated QA process, in which eValid's eV.Manager is used to run a complete library of scripts to make sure everything is as it should be.

eValid Product Review (Spanish)

In response to a query on a blog titled "Because Your Son Could Be A Tester", Mr. Raul Sanchez Echeverria posted a very nice eValid Review (Spanish). It's a positive review, but we do think that he missed a couple of points. Here's what we posted on his Blog to augment his comments:

• It turns out that normal-page sync is built in and automatic — using 6+ different page complete signals, and this includes frames. For AJAX applications there exist eValid commands to do that work — but they require some knowledge of the AJAX application.
• Interaction with JavaScript is pretty much automatic, also, but you need to turn "mouseover recording" ON to get the best results. Moreoever, if that is insufficient there is a family of built-in DOM manipulation commands that give you direct access to each element on the page by its DOM characteristics.
• On comparing images, did you not see the Validate Selected Image command? It records the checksum of the image so you don't have to save the images and compare them separately (but of course you can do that if you wish).
• LoadTest runs are limited by RAM and if you use the THIN playback engine you can get 200+ per PC image. It is possible to consolidate all of the data from multiple PC images, and there is a utility to do that for you, but in LoadTest runs the main thing is to impose load on the server.

Thanks once again to Raul for his deligent work.

Backbase/Explorer Demonstration

The AJAX environment supported by Backbase includes an example called the Backbase Explorer, which illustrates key features of how the Backbase system works. Our Backbase/Explorer Example & Demonstration Script shows hoe eValid can be used to test this example. Our workup uses both a DOM-based playback synchronization step, and a series of position-independent (index motion) commands.

eValid Maps Live AJAX Example Illustrates Motion Command Operation

The Microsoft Maps Live site is a sophisticated AJAX application that has some powerful features that are very difficult to test.

As an illustration of how well eValid handles the Maps Live application here is an example script description, Microsoft Maps Live AJAX Testing Demonstration. The writeup includes a description of the properties of the application, what makes it difficult to test, and how we approached the test script development. There are two examples:

• Script #1 illustrates searching and validating within the Maps Live system.
  
• Script #2 shows how eValid searches for a particular location and then shows a particular map sequence indicating where that location is.

The most important feature of these examples is that in playback mode (as might be used in monitoring) both examples are entirely Desktop Safe scripts without requiring use of Lock/Unlock commands.

Production Monitoring Success Story

Here is an example of how one of our customers has been using eValid to perform Rich Internet Application (RIA) monitoring websites. As explained in our Production Monitoring summary, the offering provides detailed Quality of Service (QoS) to a variety of commercial clients.

Commercial Licenses for eValid Available

eValid has a license option that gives a corporate user a very flexible way to apply eValid operations in a wide range of commercial activities. An eValid Commercial License runs on a web-authenticated basis and can be configured for any combination of eValid features.

3rd Annual Conference of the Association of Software Testing (CAST 2008)

Scheduled for 14-16 July 2008, Toronto, Canada this is the third rendition of AST's conference, largely engineered by Cem Kaner Ph.D., Professor, Author, and Attorney. CAST 2008 features keynotes by Jerry Weinberg, Bob Sabourin, and Cem Kaner.

eValid V7 Support Ended

As eValid users are aware, support for prior versions of eValid generally ends a few months after the new version is introduced. We give users plenty of notice that support is going to end. Support for eValid V7 has now officially ended. All technical issues with users of V7 are automatically referred to the V8 level and an upgrade will be required.

Technical Resources

Here are some useful technical resources for eValid users:

• Take a few minutes to watch some of our Annotated/Animated Movies. You'll find seven that show how eValid works from the GUI level, and three that deal with specific testing situations.
  
• If you run into a tricky problem testing a website, you might consider checking out the 40+ entries in our Worked Solutions Resource. You'll find a wide range of already-worked examples showing how eValid solves typical testing situations.
  
• For general eValid usage you may wish to check into the set of our "top 99" Power User Tips & Tricks. They cover everything from simple functional testing to detailed LoadTest experiment development.
  
• If you have a technical issue with eValid remember that you can use the Submit Script to eValid option in the Help Pulldown option.

If you wish, you can post a comment on this Blog entry and we will answer your question here and/or send you an email.

Prior eValid New and Activities

In you are interested in the pre-BLOG material about eValid please go to
this Web Quality News: Updates & Activities.

eValid Operation with Morfik-Developed AJAX Apps Confirmed

The Morfik WebOS AppsBuilder is a powerful system that creates an AJAX application able to run on any JavaScript browser using libraries and support facilities within the product. Morfik claims that WebOS AppsBuilder applications are Web 2.0 ready right out of the box.

As an illustration of how well eValid handles Morfik app operations, here is an example script description, Testing The Morfik Development Environment. The writeup includes the complete eValid scripts for the Morfik home page -- an application that is developed entirely with their OWN AppsBuilder system. An important feature of this example is that it includes specific AJAX synchronization logic that uses built-in eValid features.

1st International Workshop on Testing Database Systems: 13 June 2008, Vancouver, Canada

How do you really test databases? Here's a workshop at which the goal is to devise new techniques to reduce the time and cost to test database products, and that clearly has an impact on "things web." Full details at DBTest 2008.

eValid Operation with IE8 Beta Confirmed

Microsoft has released a "Beta 1" of the new IE8 browser and eValid users will be pleased to know that eValid technology works perfectly well with the IE8 browser installed.

Download Timing Success Story

Here is an example of how one of our customers used eValid to measure over the web FTP download time. In this application, tests were run on a regular basis over several months to accumulate statistics on how long a ~10 MByte game executable file would require to download over a standard high-speed DSL connection.

As described in our Download Timing summary, the customer was able to pinpoint a range of infrastructure problems and load-balance the delivery support of their application.

Rich Internet Application (RIA) Monitoring

eValid is capable of simulating ANY kind of user interaction, including some of the most complex transactions that can be performed on a web e-commerce application. Running full-browser, completely stateful tests is a sure way to confirm that your website is delivering what it is supposed to deliver. We offer RIA Monitoring Services and we can also outfit your internal RIA monitoring setup with our RIA Monitoring Standard Package.

eValid V7 Support Ending; V8 Upgrade Urged

As eValid users are aware, support for prior versions of eValid generally ends a few months after the new version is introduced. We give users plenty of notice that support is going to end.

3rd Workshop on Automation of Software Test (AST'08): 11 May 2008, Leipzi, Germany

The AST2008 workshop, also part of the International Conference on Software Engineering (ICSE'08) focuses on the automation aspects of software testing, from a theoretical and academic point of view.

6th Workshop on Software Quality (ICSE 2008): 10-18 May 2008, Leipzig, Germany

Sixth in a continuing series, of teneral interest to Software Quality people worldwide. Part of the International Conference on Software Engineering (ICSE'08) Complete details on the workshop: WoSQ2008.

Technical Resources

Here are some useful technical resources for you:

• Take a few minutes to watch some of our Annotated/Animated Movies. You'll find seven that show how eValid works from the GUI level, and three that deal with specific testing situations.
  
• If you run into a tricky problem testing a website, you might consider checking out the 40+ entries in our Worked Solutions Resource. You'll find a wide range of already-worked examples showing how eValid solves typical testing situations.
  
• For general eValid usage you may wish to check into the set of our "top 99" Power User Tips & Tricks. They cover everything from simple functional testing to detailed LoadTest experiment development.

eValid Operation with GWT "Kitchen Sink" Confirmed

The Google Web Toolkit (GWT) is a popular method for creating strong user-interactive AJAX applications with embedded JavaScript that is actually created from Java sources. The end-user tests of these AJAX applications has to be "in the browser" and this is where eValid comes into play.
As an illustration of how well eValid handles GWT operation, here is our complete Testing AJAX Applications -- GWT Kitchen Sink Example. The writeup includes the complete eValid scripts for three different example applications.

eValid Partnership Offer

If you are a contractor/consultant, or if you are involved in a web application development firm, we want to hear from you! There are many ways we can cooperate in delivering eValid's unique capabilities. Here is an outline of some of the Partner Opportunities. Please contact us ASAP if you're interested!

Remote Measurement with eValid

Here's a summary of a Remote Measurement & Reporting project that we did for a customer. The goal was to measure response time of a web server in delivering a lengthy page. The question asked was, "how long does it take for the whole page to show up, AFTER the initial part of the page appears?" The eValid test used a series of SyncOnText commands to establish the timing points, then ElapsedTime commands to get the data.


This project summary is one of a series of eValid Success Stories. Let us know if you need to do something we've already done!

Monitoring Success Story

Here is an example of how one of our customers is using eValid to perform hundreds of monitoring tests per hour -- over 1,000,000 tests per year. In this application, tests are applied to a Transportation System Portal at intervals between 10 minutes and 60 minutes. Results from each playback are parsed and delivered to a central monitoring and reporting system. Experience over time suggests that problem detections (availability, slow response, etc.) are in the 1%-2% of tests run -- a result due to the very close tolerances required of the monitored website.

Automated Technology for Verification and Analysis: 20-23 October, Seoul, South Korea

The ATVA2008, is the sixth in this series of thes workshops, this time held in South Korea, and tends to focus on higher-level aspects of automation.

eValid Partnership Offer

If you are a contractor/consultant, or if you are involved in a web application development firm, we want to hear from you! There are many ways we can cooperate in delivering eValid's unique capabilities. Here is an outline of some of the Partner Opportunities. Please contact us ASAP if you're interested!

ICWE 2008 Conference: 14-18 July 2008, Yorktown Heights, New York

The ICWE 2008 Web Engineering is going to be of general interest to workers in the web area, but it might be of interested to testers, too!

V8 Complete Documentation Available

The recently released eValid V8 technical documentation is now available for use by eValid users in PDF format. The new PDF User Manual runs to over 890 pages (about 21 MBytes). If you're an eValid user and want to have your own copy, please let us know using the eValid Question page, and we'll email you instructions for the download.

4th Workshop on Advances in Model Based Testing: 9-11 April 2008

The A-MOST 2008 sub-conference looks into the sub-area of model-based testing, as part of the larger Lillehammer, Norway event.

First International Conference on Software Testing, Verification and Validation: 9-11 April 2008, Lillehammer, Norway

ICST 2008 is a new conference aimed at the intersection of academic and industrial development.

New Index Motion Commands Available

We've added two commands to the powerful DOM Element Manipulation/Motion command set:

• IndexValidateObjProperty -- checks whether the current sourceIndex has a particular value. This assumes you've already used an IndexFind command to locate the element you're looking for, and now you want to confirm that the specified property of that element has a particular value.
  
• IndexSaveObjProperty -- saves the value of the current sourceIndex in a file for later comparison/validation. Similar to the above, you can print out the current value of the selected element's named property for separate confirmation/validation.

This capability effectively extends eValid DOM manipulation to include validating page elements entirely independent of page structure. Tests constructed with this approach are nearly immune to dynamic page changes!

Web Testing Workshop (July 2008)

This workshop may be of interest to our readers because it focuses now (in a change from the past) on EVERY aspect of web application testing and validation: Workshop on Testing, Analysis and Verification of Web Software. This highly specialized workshop is being held in conjunction with ISSTA 2008, also a well known and highly reputed conference of many years running.

PerformanceTest Success Story

Here's a summary of a PerformanceTest Project that we recently completed.

The main feature of this project was use of the eValid test engine to analyze the capacity of an industry leader's server complex, to provide confidence just prior to a big promotional effort.

To do this we created a set of eValid scripts that closely approximated "from the field" data about how the web application was used. Then we ran a series of tests to help pinpoint performance shortcomings, so that the web hosting team could make critical changes before the release date. This prevented a PR nightmare and serious loss of income for the client.

This writeup is one of a series of eValid Success Stories.

Desktop Safe, Dynamic Page Adaptive, Playback Method

If you have a website where the pages are dynamic -- so that the indexes of where things are on the page change around from day to day -- you may have a situation that is beyond the ability of the eValid Adaptive Playback feature to compensate. For example, if you had the following command in your script:

FollowLink 0 503 "" http://www.website.com/servlet/copy_asset?id=3D33221875
""

this could be replaced with the the 2-command sequence:

IndexElementFind 0 DOWN "nameProp" "uniqueName" "" IndexFollowLink 0
""

Alternatively, you could also use this 2-command sequence:

IndexElementFind 0 DOWN "nameProp" "uniqueName" ""
IndexElementClick 0 ""

You can use the eValid PageMap feature to identify the appropriate values for nameProp and uniqueName.

eValid V8 Released

There's a brand new version of eValid out: eValid V8. Some of the main features of the new release are:

• Addition of over a dozen new features (including the EPI, described below).
  
• Advanced support for AJAX application playback synchronization.
  
• Revised and updated site analysis report look and feel.
  
• Special support for "spoofing" any kind of browser, and special commands to support server load balancing experiments.
  
• Options to help customize playback event logs for monitoring, and new interfaces to popular monitoring environments.
  
• New product bundles and new pricing structures. Here's a quick summary of the New Features you'll find in eValid V8.

Here's a quick summary of the New Features you'll find in eValid V8.

Large Site Analysis Scans Offered

eValid evaluations have limits, and when you are doing a site analysis on an EVAL license you are limited to 100 pages or 500 links. While this may be enough so you can get a feel for how eValid works, it may not give you a feel for how it performs on a truly LARGE site.

An example of what we mean by "large" is this December 2007 Swiss Army WebSite Scan. It's a bit over 15,000 pages, and over 300,000 links!

If you'd like to see how eValid works on your own site, we can run a similar "large scan" for you and publish the results. Let us know the website you'd like us to scan using the eValid Question Form. We'll let you know as soon as the scan is live.

JAVA, PERL, Visual Basic (VB) Programmer Offer

Very soon you'll be reading about the new eValid Programmatic Interface (EPI), being released as part of the newest eValid system V8 update. The EPI exposes the entire eValid product as a resource for programs that can drive the eValid browser directly. Here ia a Technical Specification of this capability.

Our implementations are all in C++/CPP, and we know that environment very well. We slso know we don't have enough skill in some other programming languages. So, we are looking for a few skilled JAVA, PERL, and Visual Basic (VB) programming gurus who can help us convert a few pre-worked C++/CPP programs into the equivalent behavior in JAVA, PERL, and VB programming contexts. In exchange for your conversion work we will provide you with a complimentary license for the new eValid V8 "Programmers Bundle."

For full details on how to get in on this offer, please throw your hat in the ring using our eValid Question Form. Be sure to mention the eValid "EPI Programming Offer", and don't forget to say whether you are a JAVA, PERL, or a VB guru!

EPI Description Released

A major part of the new eValid V8 release is the addition of a fully accessible eValid Programmatic Interface (EPI). This new interface brings eValid as a test engine within the scope of popular programming languages, via a COM-based DLL that exposes key features of the eValid test engine. Here's a summary of the main EPI Features.