Friday, May 13, 2011

Results From A Small Experiment

Previous test reports pointed out how eValid pretends to be a particular smartphone, looks at how several different websites behaved in delivering material to that smartphone client browser, and studies the differences in the responses by the Amazon website to different smartphones.

Next, we tried a simple experiment with a smartphone application that uses AJAX. We wanted to see how the use of AJAX in an application affects smartphone behavior. For our test, we chose a beta-status application of a regional transportation system that offered real-time schedule information delivery to smartphones.

The experiment involved a simple test: On the mobile application, eValid pretending to be an IPhone 4 browser, request schedule data, synchronize final delivery, and measure the amount of time that delivery took. We found from static tests of that script that the data download on a per-test basis was about 330 KBytes.

The LoadTest script was set to start Browser Users (BUs) off at a constant rate until a total of 100 was reached. The repetition count for each BU was set high enough so that at the end of the test all 100 BUs were repeating the test -- all independently. The timing tests were very interesting. The LoadTest Data Report suggests that the response time for this application degrades by a factor of about 5:1 when the number of simultaneous requests grows above 75.

(Please note that this experiment was done to illustrate eValid capability; we're not making any recommendations to the agency actually responsible for the application implementation.)

You can read about our short experiment in detail at this page: Mobile Agent Test Page -- Loading Experiment #1.

No comments: