Testing Mobile Web Apps

An eValid user wrote:

Can you please explain in more detail how eValid simulates web applications on mobile devices? Also, you claim that your tests done by eValid when simulating mobile devices are functionally equivalent to using the mobile device? Can you elaborate?

Background
First, some background information and details... The underlying question here is, what does it mean to test a mobile web application?

It is important to make sure of the distinctions between how a mobile web application responds and how a mobile device responds. In other words, you have to ask yourself: are you testing a mobile device; or, are you testing an application that runs on a mobile device.

If it is the latter — testing a mobile device — then eValid is not well suited to that type of work. eValid is not a device tester — it is a web application tester.

But if it is the former then the question really reduces to, "what does it mean to test a mobile web application?"

eValid Solution
Being primarily a web application testing system the main focus for eValid is, from a user's perspective (that is, from inside a browser), how does the application behave? The key word is "behave," and to eValid that means the kind of content is delivered (is it correct or not?) and how fast is it delivered (does it meet performance thresholds).

When you use the eValid SetUserAgent command to set up eValid to act like a mobile device, the data eValid generates in response to requests from the server is no different from what the actual device generates from the server. In other words, if you change the "identity" of the browser as a test driver, you change the response from the server, and you effectively test the server's ability to support the "identity" of the browser.

And the key facts about quality — validation of response data and quantification of response times — are perfectly measurable from the eValid desktop. In most instances, mobile device download speeds are nominally in the same range as those for most desktop applications. The data delivered to a mobile device passes through a wireless gateway of some kind before reaching the device, whereas the same data has to pass through some kind of internet service provided before it reaches your desktop.

In other words, you don't need the actual device to stimulate the server responses, and you don't need the actual device to validate content and to measure performance timing data.

_________________

eValid Support

Recent eValid Technology Questions

Recently there have been a number of eValid User Forum postings that raise interesting questions about eValid's architecture and underlying technology.

• Can eValid analyze any web page? -- Are there limits to what kinds of pages eValid can analyze?
• Does evalid have an API? What is it like? -- Details about eValid's Programmatic Interface
• eValid operating with Web 2.0 and AJAX applications -- Pointer to the Business White Paper that sumarizes eValid's development history.
• Any Kinds Of Web Applications -- What eValid can and can't test?
• IE Browser -- eValid's dependence on the IE browser: how it works!
• Drive tests using JavaScript -- Why eValid is built to never interfere with JavaScript operations.
• Why doesn't eValid work at the transport level? -- The advantages of eValid's high-level approach are explained.
• Can eValid test IPhones? -- Specifics on how to test mobile devices.

Using evalid With An AJAX Autocomplete Functionality

This scripting example investigates how eValid can be used to validate an AJAX autocomplete function. The two variants of the script shown below operate as follows:

1. Introduce a partial string into the text area, for example evali.
2. Capture the contents of the DOM after the AJAX activity has provided some suggested completions.
3. Confirm that a required string, for example evalid, is present within the available suggestions presented by the autocomplete feature.

The goal of the script is to confirm that, among all of the possible suggested completions, that the target evalid string is among the options presented. If successful, this process effectively validates operation of the AJAX autocomplete function by confirming its results with known (expected) data.

Here is the link to the full solution description: Checking AJAX Autocomplete Functionality

Manipulating the Target Window

How do you use the eValid DOM manipulation commands to modify an anchor-tag's attributes to deliver the page to a different specified new window than the one given in the HTML passage.

In this typical HTML passage if you click on the link the new page is opened in a new window named "NEW-WINDOW":

<A HREF=../URL TARGET=NEW-WINDOW>Title Of Page</A>

The key to modifying this link target dynamically is to recognize that any anchor tag that has a specified target will have the name of the target stored as one of the attributes in the defining DOM element. All you need to do is find the right element, and then put in a new string for the target page.

Here is the link to the solution description: Change the Target of an Anchor Tag Link To A New Window

Modifying DOM Elements Dynamically

How do you use the DOM manipulation commands to modify a specific DOM element dynamically?

In this illustrative solution an eValid script demonstrates the ability to modify the DOM dynamically by using eValid DOM manipulation commands to change the background color.

1. Navigates to a page
2. Changes the background color for one element to COLOR1.
3. Waits 5 seconds (so you can see the effect).
4. Changes the background color for one element to COLOR2
5. Waits 5 seconds (so you can see the effect).
6. Changes the background color back to the original background color.
7. Waits 5 seconds (so you can see the effect).

Here is the link to the full solution description: Modify a DOM Element Dynamically

Update on Mobile Application Performance

Some months ago we ran a small experiment on the Beta site of a mobile application in which we ran a small 100-BU (Browser User) test to see how performance of the application changed as the load increased, linearly, from 1 BU to 100 BUs.

As a followup, we repeated the experiment on the same application -- which now has become a full-scale "non-Beta" application -- with the results shown here:

Further Comparative Improvement of Performance Under Load

As you can see from the charts, things didn't necessarily get better, and you could say that there was some deterioration. But remember, that later data -- from over four months ago when the tests were first run -- is now from a live application. Right now it appears this application is "good" for up to about 25 simultaneous users, but performance drops off quickly after that -- as much as a 5x degradation beyond 50 simultaneous users.

Selected User Forum Posts

Beginning in mid-2010 we have directed all technical support questions to the eValid User Forum. We have learned that when one user has an issue, all users can profit from the answer.

Here is an additional selection of some of the posts that we think would be of general interest.

1. Can eValid test applications using GWT and Vaadim? -- Tricks in testing a specialzed application
2. Complex Synchronizations -- Is there a resource problem?
3. Keep Separate Cookies -- More about saving your cookies!
4. Can eValid detect graphic differences in my app's GUI? -- Methods to confirm graphics are described.
5. Use of the cache during server load test -- Details on cache locking.
6. Can you guys run a monitor on a smart phone app? -- Mobile monitoring.
7. Run a batch mode scan? -- Background site analysis.
   
8. Why does eValid base its operation on IE? -- Justifications for IE dependence.
9. Can eValid test IPhones? -- How to spoof iPhones.
10. Pretending to be a mobile client -- More about mobile client emulation.

Webinar: Structural Testing for Mobile/AJAX/Web 2.0

Run Functional/Regression Tests on Complex, Dynamic AJAX/Web 2.0/Mobile Applications Extremely Robust Tests Are Based on Structural Page Features

Wednesday, 12 October 2011 2:00 PM Eastern Time / 11:00 AM Pacific Time

When web applications often change in subtle ways, and your functional/loading/monitoring tests may begin to fail if your test scripts are "brittle". eValid's advanced structural testing commands overcomes this problem to help you achieve tests that survive page changes -- but still validate performance realistically. The eValid structural testing command set makes it possible organize tests entirely around structural properties of a page that don't change over time. A test done with structural testing methods can handle any AJAX/Web 2.0/Mobile application reliably and efficiently. Even when page structure and details (but not essential functional intent and effect) -- change substantially. Learn how to achieve greater productivity with smaller budgets, maximize your IT investments, and get more work done in less time with less energy. Build reliable, robust AJAX/Web 2.0/Mobile tests once -- and you won't have to worry about them again.

Webinar Outline

Introduction to eValid's Architecture. Functional Test Creation: "What you see and do, is what eValid records, is what eValid reproduces". Methodology Overview: Record From Life, Adapt From Page Facts Index/Motion (Algorithmic/Structural) Command Summary Typical Script Passages: "Recorded From Life" and Structural Versions Script Enhancement for AJAX/Web 2.0: Bullet-Proofing Your Playback Practical Experience & Recommendations

You are cordially invited to attend this free Webinar. REGISTER NOW!