How eValid Scales

There has been a lot of discussion in various forums -- including in our own eValid User Forum -- about how to scale tests up from singleton runs to large load tests. The goal seems in all cases to be to preserve the precision and accuracy of an eValid playback but still run enough evalid's in parallel to yield a realistic load.

The natural questions are, when you increase the number of eValid instances, what resources are used up first and what limits are hit along the way. Here are some of the main points that contribute to this discussion, in order of where they form limits:

1. Initial RAM Usage
   When eValid finishes loading -- even when the initial page is about:blank the footprint is about 12 MBytes. That means that an initial load of 100 BUs will consume 1.2 GBytes of RAM. Which is certainly feasible with a 3 GByte machine.


2. RAM Usage Growth
   As each browser continues working its footprint grows because the browser stores in its workspace all of the JavaScript files and a lot of other material that it picks up along the way. The longer the test the greater the growth. We have noted footprints for very long tests as much as 100+ MBytes per BU. (This behavior argues for smaller tests and/or for more-often restart of the BUs in your load test run.)


3. Cache Space
   As your tests run on and on, with more and more pages, there are more pages stored in the cache. In LoadTest runs, though, this doesn't matter as much because you typically don't use cache at all. (Periodic DeleteCache instructions are a good idea.)


4. Virtual Memory Size
   There are special instructions that come with eValid to make adjustments to the Virtual Memory size, etc. This is not a fixed number, but when you do the "machine adjustments" on most Windows systems you ask Windows to use the best size of the paging files to the maximum available.


5. GDI Object Count
   You can't control the GDI object count, but we know (from our conversations with Microsoft technical support) that you may run past the built-in Windows limit of between 256 and 65,536 GDI objects per user.


6. Input/Output [I/O] Bandwidth
   This depends a lot on the web application you are testing. Some are very economical of I/O capacity, and others are very hungry. The only way to make sure this is not a limit is to look at the PerfMon data in the Task Manage to monitor the networking activity.

There are some other tricks to maximize the number of BUs per run -- tricks that we use to get over 10,000 BUs off one machine. That'll be a subject for a subsequent BLOG post.