Monitoring Performance Case Study

eValid seems to be used increasingly for monitoring applications because it is a stateful, realistic, efficient and reliable engine for collecting user-oriented response-time data.

In monitoring mode, the Windows scheduler launches one or more eValid playbacks via the batch interface, and the data that's generated shows up in the network monitoring environment. We have standard PERL-based integrations with Nagios, Nagvis, GroundWork, Springsoft, Hyperic, Zenoss, etc.

A common question is: How do I size my machine to get the most eValid output?

Here's a case study that gives you some idea of what you might get, based on recent inputs from one of our customers.

The machine being used to drive the monitoring is a 2.8 GHZ dual-core Pentium with 4 GB of RAM running in Windows 2003 Server (SP2). The machine has been "adjusted" to modify the OS internal performance priorities, adjust the heap space, and adjust the virtual memory sizing parameters.

The scripts being by the eValid instance run are fairly typical of functional tests used in monitoring mode. They break down as follows: (a) ~20% "simple", being just visits to one or two URLs; (b) ~60% "typical", involving a login and post-authentication visits to several pages; and, (c) ~20% "heavy", meaning they involve playback into detailed AJAX pages that involve many of eValid's more-advanced features. Most of the tests can be run in parallel -- as many as 8 in parallel, in practice -- but some of the tests have interactions that require them to be run serially.

CPU utilization of about 40% happens when this one machine is running about 600 tests per hour, all re-run at a 5-minute interval. This totals out to about a half-million tests per month.

At 600 tests per hour and at 40% utilization we think this machine is pretty much at its limits. Pushing any more runs the risk of not having enough extra time available in case one or more or many of the tests start to take longer.

All in all, it is a pretty impressive result.