Monitoring Performance Benchmarks

When you run a functional test in "monitoring mode" you typically report the test results to a network reporting system (e.g. Nagios, or GroundWork, or Hyperic, or Zenoss, or some other similar system). One of the questions we often get after the reporting integration is done is this: "How many tests CAN you run with eValid on one machine per hour?"

We just heard from one of our customers who is using eValid on four agent machines, and he reports that the aggregate effect of his four agent machines is over 1,800,000 tests per month.

This works out to about 625 eValid playbacks per machine per hour, every hour, 24x7x30 -- every day of the month.

Clearly he has to use multiple instances of eValid on each agent machine to get this kind of result. Given that the typical test he runs takes one or two minutes, this particular installation achieves a parallelism of about 10:1 or more on a continuing basis.

The CPU utiliztion of each of these agent machines -- and our customer says that they are very hot CPUs with a lot of RAM -- runs about 40% on the average. It would probably be imprudent to try to run these with any closer safety margin.