Load Testing Mobile Web Applications

With all of the recent interest in mobile devices – including all kinds of smartphones and hand-held tablets – we decided to provide readers with a short description of eValid's capabilities in "mobile device testing".

• Technology
  The basis for the eValid approach is to simulate ANY mobile device in a realistic way. Test runs done by eValid, when it is imitating or simulating a particular mobile device, appear to the supporting web application server as if the actual device is being used. Server response times to the device being exercised in a test session – mainly intra-browser timings – are very accurate (1 millisecond resolution). The big advantage is that you can test multiple devices from a single solution platform. This makes comparing different devices' performance very easy. All you do is change the simulation specification and re-run your functional test.
• A Typical Mobile-Device Loading Experiment
  As a use-case example, we did an experiment simulating an iPhone navigating to a popular website that delivers transportation system departure times, based on the requester's location, and retrieved from a database that tracks all of the available vehicles.
  1. Using a short self-synchronizing test script that asked a very simple and standard question from the database: "When is the next vehicle departure?" The script recorded a single data point: the time the database takes to retrieve the data to the user. For this application this is a Key Performance Indicator (KPI).
  2. We ramped the load from 1 Browser-User (BU) to 1,000 BUs by adding BUs at 1-second intervals. It took about 16 minutes to reach the 1,000 level. Once there, we held the imposed load at 1,000 BUs, all repeating the same test script as rapidly as possible for another 15 minutes.
  3. We used an Amazon cloud-based "Quadruple Extra Large" machine as the test driver. During the experiment the driver machine had only about 15% CPU usage and < 5% network usage at the full 1,000 BUs level. Overall, we collected data from over 5,900 KPI timings during the entire run.
  4. Here's the chart that shows the KPI vs. imposed load: 1,000 BUs Chart You can see from the chart that response time is constant up to about 200 users and then grows linearly with the increasing load.
• Detailed Example Explanation
  Here is the detailed Experiment Description that describes all of the above in complete detail, and also includes the actual eValid test script used.
• Webinar YouTube Movie: "Load Testing Mobile Web Apps"
  Here is the webinar movie associated with this experiment: Mobile Testing Web Apps [Webinar, 39:07].

You may also want to see the earlier eValid Blog item that deals with related aspects of testing of mobile web applications: Testing Mobile Device Web Applications.