Measuring [email protected]’s performance impact

THERE ARE A FEW legitimate reasons not to run Stanford’s [email protected] client. You could be without an always-on Internet connection, or barely able to scrape together enough change to make your next utility bill payment. Heck, you may even think that searching for aliens is a better way to spend your spare CPU cycles, and that’s your choice. However, some have raised questions about whether or not the [email protected] client has an impact on overall system performance, something that could prevent individuals and especially businesses from running the client on their machines.

Of course, running the [email protected] command line client isn’t supposed to take away system resources from more important processes. The client itself is tagged with a low process priority when running in Windows, so just about any other system process should have first dibs on system resources. [email protected] should only use CPU cycles your system would otherwise leave fallow, which means there should be no perceptible impact on performance when running the Folding client.

Despite that fact, some businesses may fear a loss of computational productivity, and gamers may want to avoid a potentially deadly drop in frame rates, just to be safe. Rather than simply trusting that [email protected] doesn’t impact system performance or assuming that running the client will slow things down, we’ve run the client through a gauntlet of tests to set the record straight, one way or another. Read on to find out just how much of an impact, if any, running [email protected] will have on system performance.

[email protected]?
So what’s this [email protected] stuff all about? I’ve lifted a helpful little primer from TR’s official [email protected] team page:

Folding at Home is a distributed client computing effort by Stanford University intended to help understand how proteins assemble or “fold.” Exactly how proteins assemble themselves is a mystery, and why the proteins sometimes fold improperly or “misfold” is also a mystery. Quite a few serious diseases are related to the misfolding of proteins, such as Alzheimer’s and Parkinson’s disease, to name two. By donating your CPU’s spare cycles, you are contributing to the effort to understand how the proteins fold, which is the first step to understanding how basic proteins work and how we might treat these diseases.

Our preferred method of running [email protected] is to use the text-only command line version of the client, which can be run transparently as a service, using FireDaemon, for any Windows NT/2k/XP-based PC. [email protected] can also be run on Apple’s OS X and on Linux, but for the purposes of this article, I’ll just be covering the Windows version of the client.

If you use Windows 9x, or if don’t want to run [email protected] as a service using FireDaemon, you can always run the text-only client manually. In fact, you can even copy a shortcut to the client into your Windows Startup folder so that it loads automatically each time you start Windows. Running the client as a service is a cleaner way to do things, because it will let you effectively hide the client from the hands of meddling users. The service will also automatically restart the client should it crash or shut down for whatever reason.

Now, let’s get on with our testing.

 

What to watch for in the test results
Ideally, the graphs we’re about to see are going to be very boring.

Sorry.

[email protected] is supposed to run as a low-priority process, which means that Windows will give just about every other program, including our benchmark applications, priority when allocating resources. The differences in benchmark scores between systems running with and without the [email protected] client should be negligible, and we’ll find out if that’s really the case.

Because [email protected] runs as a low-priority process, it’s not going very productive if you have other programs eating up your system’s resources. In this case, [email protected] will still run in the background, but it won’t crunch through many work units, since the client will be running on whatever table scraps Windows throws its way.

Remember: we’re not out to measure how well a machine folds, but how transparent a [email protected] client can be running in the background. The less of an impact [email protected] has on benchmark results, the less a user is likely to experience any kind of slowdown in day-to-day computer use.

Our testing methods
As ever, we did our best to deliver clean benchmark numbers. Tests were run three times, and the results were averaged. Our test systems were configured like so:

  High-end Low-end
Motherboard Albatron KX400+ Pro
Processor AMD Athlon XP 2100+ AMD Duron 1.2GHz
Front-side bus 2x133MHz 2x100MHz
Chipset VIA KT333
North bridge VIA VT8367
South bridge VIA VT8235
Memory size 512MB (2 DIMMs) 256MB (1 DIMM)
Memory type Corsair XMS3000 PC2700 DDR SDRAM
Graphics NVIDIA GeForce4 Ti 4600 ATI Radeon 9000 Pro
Graphics Driver Detonator 40.72 CATALYST 2.23
Storage IBM 60GXP 40GB 7200 RPM ATA/100 hard drive
Operating System Windows XP Professional SP 1

The high and low-end systems I’ve set up for testing should give a pretty good cross section of what people are actually running today. I suppose I could have tested something even slower to simulate the kind of decades-old hardware that some workplaces impose on their employees, but honestly, some of the benchmarks probably wouldn’t even have run, with or without [email protected] running in the background.

The high and low-end systems were run with and without the [email protected] client crunching in the background as a service via FireDaemon. After each round of tests with the [email protected] client activated, the client’s log was checked to ensure that it was indeed crunching on a work unit during the test and not otherwise inactive or waiting for a work unit from the server.

Testing was limited to Windows XP Professional, but the results should hold true for Windows 2000, NT, and XP Home, all of which can prioritize active processes. If you are running Windows 9x, you may not see the same kind of impact on system performance, but that’s what you get for running an operating system based on DOS.

We used the following versions of our test applications:

During testing, the display resolution was set at 1024×768 with 32-bit color at a refresh rate of 75Hz. Vertical refresh sync (vsync) was disabled for all tests, and all of the 3D gaming tests used the highest detail image quality settings in 32-bit color.

All the tests and methods we employed are publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.

 

Memory bandwidth

Memory bandwidth can often be a determining factor for overall system performance, but as you can see, the [email protected] client has no apparent impact on available memory bandwidth in either of our test systems.

Business performance

Business Winstone performance should be especially important for businesses since it tests performance in word processing, email, and web browsing applications. Oddly enough, our high-end system actually scores a little better with the [email protected] running in the background, but the score delta is within the margin of error. Our low-end system scores a little lower with [email protected] running than without, but again, the difference is small enough to chalk up to the margin of error. Really, half a point in Business Winstone is nothing to get too excited about.

In last year’s Content Creation Winstone, the [email protected] systems are a little slower, and the difference in performance here is a little bigger than what we saw with the Business Winstone test. Still, it’s a close race.

The performance gap between the systems with and without the Folding client is even smaller in this year’s edition of Content Creation Winstone.

 

3DMark2001 SE

Like our business and content creation tests, 3DMark2001 SE shows [email protected] having only a very small impact on overall performance for both high and low-end systems.

Quake III Arena

In Quake III Arena, [email protected] again has a miniscule impact on performance. At lower resolutions, the [email protected] systems are only a few frames per second slower than systems running without the client enabled. Quake III Arena is a little old, so let’s throw a couple of more recent games into the mix.

 

Jedi Knight II

The relatively flat performance profile of Jedi Knight II suggests that it’s a lot more limited by overall system performance than by the graphics card, at least until we start hitting high resolutions. Throughout the different resolutions tested, [email protected] has a negligible impact on frame rates, which means that no one’s going to buy your “folding lag” excuse for getting owned in light saber matches anymore.

 

Comanche 4 Demo

Like Jedi Knight II, the Comanche 4 demo seems more limited by overall system resources than the graphics card up to a resolution of 1600×1200. And, just like Jedi Knight II, the impact of the [email protected] client on frame rates isn’t going to be big enough for anyone to notice. Seeing the difference between 30 and 60 frames per second is one thing, but there’s no way anyone is going to be able to see a difference of 0.3 frames per second.

 

SPECviewperf

In SPEC’s viewperf suite of 3D workstation tests, [email protected] again has a negligible impact on overall performance. About the only interesting result here is that the Radeon 9000 Pro on our low-end test system has problems completing the Unigraphics (ugs) component of the viewperf suite and registers a score of zero regardless of whether or not the [email protected] client is running.

 

POV-Ray ray tracing

In our POV-Ray rendering tests, both the relatively simple ntreal and more complex glasschess scenes show [email protected] has little impact on trace times. Overall, the high and low-end systems running the [email protected] client are only about 1% slower, if that.

Media encoding
For better or worse, legally or not, media encoding is something all the kids are doing today. Will [email protected] slow you down?

Nope, at least not when encoding a 60MB WAV file into an MP3 with LAME 3.89. You’ll notice that the bars don’t quite perfectly line up in our graph; that denotes a fractional difference in performance that isn’t reflected by our bar labels, which are set to display no decimal places. For all intents and purposes, MP3 encoding with [email protected] running in the background is no slower than without.

Moving to DivX, we see a noticeable difference in encoding performance. I say noticeable not because you’ll actually notice the difference in encoding performance, but because our DivX test takes several minutes to complete, so the impact of having the [email protected] client running ends up being a couple of seconds on both our high-end and low-end systems. Overall, those few seconds work out to less than a 1% difference in overall DivX encoding performance.

 

ScienceMark

ScienceMark continues a trend we’ve see throughout testing: running [email protected] simply doesn’t have much of an impact on overall system performance. Let’s break ScienceMark down into its individual tests to see if we can find some variation, somewhere.

Nothing to see here; move along. [email protected] isn’t stealing enough resources from even ScienceMark’s computationally-intensive tests to make much of a difference in overall scores.

Sphinx speech recognition

Our Sphinx speech recognition tests bring me to the end of this broken record of benchmark performance. Neither the high-end nor low-end systems experience more than a negligible dropoff in performance with [email protected] running during our Sphinx speech recognition test. At this point, we should be expecting nothing less.

 
Conclusions
The results of our testing couldn’t be clearer, at least for the systems we’ve tested today. Quite simply, the impact of running [email protected] in the background is negligible. Even the most discerning users won’t notice it. Windows allocates CPU time slices according to process priority, and as long as the [email protected] client is running as an idle-priority task (which is how it runs by default), one shouldn’t notice any slowdown. Remember that we’re also running FireDaemon in the background, and the impact of this small service management tool is also negligible.

With the results we’ve seen here, one more exuse for not donating excess CPU time to the [email protected] project is gone. Whether you’re playing games, encoding media files, surfing the web, slaving away in a cubicle at work, or mumbling to your computer, running the [email protected] client won’t slow you down. To effectively fold, however, you will need an always-on Internet connection. Also, expect to pay a few dollars a month extra on your electric bill as your system’s CPU crunches away at full tilt all the time. In my book, those are small sacrifices, especially if you count completing a work unit as your good deed for the day.

So why not head on over to Stanford’s [email protected] page and download the client? While you’re at it, you can also join TR’s very own folding team, and help make the world a better place, one work unit at a time. 

Comments closed

Pin It on Pinterest

Share This

Share this post with your friends!