Scaling Issues in Mobile Testing

The number of different mobile devices keeps growing. According to a Ryan Lawler post, some 7,000 Different Types Of Mobile Devices Access Facebook Every Day . Daily. And this was in 2012. The total number of combinations that may need to be tested is surely growing.

Two factors are involved: the number of different browser versions (currently > 500) and the number of different device development environment versions (currently > 100). You have to multiply these two factors to get the total "perfect" test count. Two factors are involved: the number of different browser versions (currently > 500) and the number of different device development environment versions (currently > 100). You have to multiply these two factors to get the total "perfect" test count.

Hence, scale is clearly important if you are working on testing mobile web applications. With that many device-browser combinations it becomes quite difficult to obtain actual instances of all of these devices. While some service solutions we've seen offer direct device tests of several hundred combinations, using actual devices that are controlled remotely, device-browser combinations in the 1,000's or 10,000's just aren't feasible economically.

eValid can overcome a major part of that difficulty with its ability to imitate other any browser. eValid can do this when launched from a command line option, or it can do this in individual subwindows launched inside a playback script using the SetUserAgent Command. The result is a quick -- and highly scalable -- method to test a web application in a variety of browsers.

The behavior variations between browsers can be quite remarkable, as this Mobile Agent Demonstration Experiment shows. For these 20 different mobile device browsers it's important to note how the pages vary, and how big the variations are in the downloaded file size. For good measure, here are Selected Screenshots for 200+ Mobile Devices.

Using eValid's device imitation feature it's possible to run full-realism, dynamic tests on 1,000's of browser variations by just putting the playback process in a loop, each repetition trying a different setting. Web application validation — an otherwise very difficult task to to with 100% thoroughness — becomes much easier because you can use a single test appliance with great scaling leverage.

Customer Finds Mobile Performance Flaw

We heard from an eValid user about a novel use of the internal timing capability to track down and fix a problem that only appeared when the application was viewed with evalid imitating a mobile device.

Their testing process involved setting up timing measurements within the DOM using the structural commands. The timing data let them to identify the fixes that were needed, and the same test setup was used to verify the performance improvement that was evident on the mobile device.
Here is the writeup they provided describing these results:


Click image for full size version

Quality Week Proceedings Available

The complete conference proceedings for the QualityWeek and QualityWeek/Europe conferences - from 1998 to 2002 - are now available for download. 

The files below are the complete contents of the Conference CDs unified into a single PDF document.

Caution: The PDF are big files, this may take some time to download.

1999	QW1999CD.pdf -- 32 MB -- [1,816 pages] QWE1998CD.pdf -- 20 MB -- [1,251 pages]
2000	QW2000CD.pdf -- 82 MB -- [1,712 pages] QWE2000CD.pdf -- 147 MB -- [1,712 pages]
2001	QW2001CD.pdf -- 72 MB -- [2,070 pages]
2002	QWE2002CD.pdf -- 100 MB -- [1,300 pages] QW2002CD.pdf -- 65 MB -- [1,783 pages]

Selected Recent Forum Posts

Here are some selected posts from our eValid forum:
 
eValid's Pagemap vs Chrome's inspect element -- They reallly are two different ways to do the same thing, only eValid's Pagemap is more focused on testing aspects.

Using eValid for a Facebook logout -- Here's a neat structural testing sequence for a tricky FaceBook logout.

Help with starting 2500 users at the same time. -- Some timing tricks to help synchronize 2,500 simultaneous users.

Monitoring Agent Question -- Distinguishing eValid Monitoring Master and Agent Stations.

Selected Recent Forum Posts

Here are some selected posts from our eValid forum:

eValid Perks -- Using the IndexFind and DOM Manipulation technology items in eValid.

Can eValid find a particular element -- What WE like best (and are most product of) in eValid!

Popup Question -- Popups are always tricky, but here is the simple procedure that always works.

eValid the Best for Monitoring AJAX Applications -- Some reasons why using a full browser makes SO much sense!

Selected Recent Forum Posts

Here are some selected posts from our eValid forum:

Why is scaling so important? -- Some practical answers that point to one of evalid's best selling points.

 eValid rendering Chrome question All about rendering, and does it make a difference or not in doing regression testing of web applications.

 Using eValid to monitor a page being hacked -- The key requirement for eValid to detect a page having been hacked is to have some kind of checksum of the page.

  LoadRunner vs. eValid question -- Why eValid's archtecture precludes the need for a "correlation capability."

eValid Component Licensing Options

With the recent completion of the primary set of eValid Patents, we have begun to set up offerings of key eValid technologies as part of the eValid Business Development activity.

Our technology covers virtually all functions needed for client-side analysis of web browser enabled applications including performance, timing, behavior, quality, security, and reliability. The technology is effectively a "one stop shop" for client-view web application testing and analysis.

If you are interested in eValid components for your own application you ought to look at the available Component Descriptions.

Please contact us directly for additional details.