Scaling Issues in Mobile Testing

The number of different mobile devices keeps growing. According to a Ryan Lawler post, some 7,000 Different Types Of Mobile Devices Access Facebook Every Day . Daily. And this was in 2012. The total number of combinations that may need to be tested is surely growing.

Two factors are involved: the number of different browser versions (currently > 500) and the number of different device development environment versions (currently > 100). You have to multiply these two factors to get the total "perfect" test count. Two factors are involved: the number of different browser versions (currently > 500) and the number of different device development environment versions (currently > 100). You have to multiply these two factors to get the total "perfect" test count.

Hence, scale is clearly important if you are working on testing mobile web applications. With that many device-browser combinations it becomes quite difficult to obtain actual instances of all of these devices. While some service solutions we've seen offer direct device tests of several hundred combinations, using actual devices that are controlled remotely, device-browser combinations in the 1,000's or 10,000's just aren't feasible economically.

eValid can overcome a major part of that difficulty with its ability to imitate other any browser. eValid can do this when launched from a command line option, or it can do this in individual subwindows launched inside a playback script using the SetUserAgent Command. The result is a quick -- and highly scalable -- method to test a web application in a variety of browsers.

The behavior variations between browsers can be quite remarkable, as this Mobile Agent Demonstration Experiment shows. For these 20 different mobile device browsers it's important to note how the pages vary, and how big the variations are in the downloaded file size. For good measure, here are Selected Screenshots for 200+ Mobile Devices.

Using eValid's device imitation feature it's possible to run full-realism, dynamic tests on 1,000's of browser variations by just putting the playback process in a loop, each repetition trying a different setting. Web application validation — an otherwise very difficult task to to with 100% thoroughness — becomes much easier because you can use a single test appliance with great scaling leverage.