Windows Vista Service Pack 1: A performance check

The story of Windows Vista is one fraught with tragedy (at least, as tragic as software can get). The operating system had a difficult birth, with Microsoft scrapping its work and starting over again about half-way through the development cycle, shedding many promised features in the process. Once out in the market, Vista had an even rougher childhood. Drivers weren’t ready in time, users perceived Vista as nothing more than a more resource-hungry version of XP with a prettier user interface, and many reviewers gave it a thumbs down.

However, as far back as January 30, 2007—Vista’s retail launch date—users disappointed with the new OS looked to a future first service pack to make everything right. Back then, nobody even knew what SP1 would bring to the table or when it would come out, but folks’ experience with Windows release cycles suggested that SP1 would fix most of Vista’s little imperfections. Some couldn’t be bothered waiting and switched anyway, finding out in the process that Vista wasn’t anywhere near as bad as everyone was saying. Others stuck with Windows XP and kept waiting.

Over a year and more than a hundred million Vista sales later, SP1 is finally out. It, too, had somewhat of a rough start, since Microsoft was forced to postpone it by about a month due to driver compatibility problems. Then, on March 18, SP1 finally appeared on Windows Update. Would it really fix everything disillusioned Vista users expected?

What’s new in SP1

A quick glance through the service pack’s quite considerable list of fixes, changes, and new features suggests that SP1 is indeed meant to improve Vista substantially—and not just by rolling all previously released updates into a single, neat package. High on the list are a handful of storage-related performance enhancements that cover copying files, extracting archives, or doing many kinds of file operations at once. According to Microsoft, SP1 is 25% faster when copying files within a single disk, and the “calculating time remaining” phase of file operations has been reduced to around two seconds.

Along with these welcome speed boosts, the new service pack brings a very long list of reliability, security, and power-saving improvements. That list is a little too long to sum up here, but users can look forward to everything from more consistent hard drive spin-downs, which should improve battery life on notebooks, to better driver stability in default drivers, and new application programming interfaces to help developers write more secure apps and better anti-malware tools.

Last, but certainly not least, SP1 introduces support for a number of new technologies and standards. There’s the new Direct3D 10.1 application programming interface, support for Intel’s Extensible Firmware Interface on 64-bit systems, and support for the exFAT file system for flash devices, just to name a few. Direct3D 10.1 should allow developers to tap into the features of AMD’s new Radeon HD 3000-series graphics processors, while EFI support should theoretically pave the way for BIOS-free PCs—at least as long as hardware manufacturers follow suit. Meanwhile, exFAT brings support for file sizes of up to 16 exabytes and more than 1,000 files per directory, where FAT is limited to 1,000-file directories and 4GB files.

We won’t devote an entire page to detailing each and every one of SP1’s improvements, since Microsoft itself takes cares of that in the official SP1 changes list. What we will do, however, is probe how SP1’s plethora of under-the-hood changes translate into performance.

Test notes

To assess SP1’s impact on performance, we performed a clean installation of the “release-to-manufacturing” (i.e. original retail-boxed) edition of Windows Vista Home Premium x64 on our test system, and we ran a first round of benchmarks with no patches and only necessary drivers applied. Then, we downloaded the x64 Service Pack 1 standalone installer from Microsoft’s Download Center, installed it, and ran our suite of benchmarks again.

Our testing methods

As ever, we did our best to deliver clean benchmark numbers. Tests were run at least three times, and the results were averaged.

Our test system was configured like so:

The test system’s Windows desktop was set at 1680×1050 in 32-bit color at a 60Hz screen refresh rate. Vertical refresh sync (vsync) was disabled.

We used the following versions of our test applications:

• WorldBench 6 beta 2
• Team Fortress 2

The tests and methods we employ are usually publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.

WorldBench

WorldBench is a good way to get a feel for how SP1 affects real-world Vista performance. This benchmark uses scripting to assess the performance of eight major Windows applications, from Firefox 2 to 3ds max 8, in a variety of different usage scenarios. We’ll start with the overall score, which rates a system based on how well it runs the entire suite of WorldBench applications tests, then we’ll look at how each app fared on its own.

Yes, Service Pack 1 is one whole WorldBench score point faster than the release-to-manufacturing version of Windows Vista. Before we draw any hasty conclusions, though, let’s first have a look at individual application performance.

Productivity and general use software

The following results cover WorldBench’s desktop productivity application benchmarks. These tests should do a good job of mirroring day-to-day usage scenarios like web browsing, word processing, or multitasking with a CPU-heavy app in the background.

Microsoft Office productivity

Firefox web browsing

Multitasking – Firefox and Windows Media Encoder

WinZip file compression

Nero CD authoring

Individual apps show more pronounced changes, but we see no clear pattern emerging. SP1 is sometimes faster, sometimes slower, and sometimes the same as the RTM version.

Users working in Microsoft Office or authoring CDs and DVDs in Nero will see slight performance benefits from SP1, but those running Firefox 2 either on its own or with a copy of Windows Media Encoder doing its thing in the background will see a slight performance decrease. As for folks extracting files with WinZip, they should see no changes.

Of course, we ought to point out that differences are smaller than 5% in all of these tests. As such, users may be hard pressed to notice any performance increases or decreases at all.

Media editing and encoding, 3D modeling and rendering

The next few tests span the more heavy duty, enterprise front of WorldBench. These probably aren’t apps your average home user is going to run everyday, but they give us a glimpse of how fancier, more processor-bound tasks fare with Service Pack 1 installed.

Photoshop image editing

Roxio VideoWave movie editing

Windows Media Encoder video encoding

3ds max DirectX visualization

3ds max 3D rendering

In media editing, encoding, and rendering tasks, Service Pack 1 performance is even closer to that of the release version of Vista. The largest difference is in 3ds max 3D rendering, with a 9-second gap between Vista SP1 and RTM. However, that only amounts to a ~1% change in total rendering time.

Gaming and desktop file operations

Team Fortress 2

A round of desktop benchmarks wouldn’t be complete without some gaming tests, and we’ve selected Valve’s multiplayer first-person shooter Team Fortress 2 for that purpose. To test TF2, I recorded a demo of myself playing as Pyro and Soldier in a ~30-player match on the game’s cp_dustbowl map, then played it back using the “timedemo” function three times for each test.

We tested at a 1680×1050 resolution with the game’s detail levels set to their highest settings. HDR lighting and motion blur were enabled. Antialiasing was disabled, and texture filtering was set to trilinear filtering only.

We see the same result in Team Fortress 2 as in our previous benchmarks. In this case, SP1 actually improves performance by a fraction of a frame per second, but even die-hard gamers probably can’t detect such a minute change.

Drive-to-drive file copy testing

For this test, we copied a 700MB video file from one 320GB Western Digital Caviar SE16 hard drive to another hard drive of the same model, and we recorded the transfer time using our stopwatch. As with all our previous tests, we ran each file copy three times and averaged the results to rule out any flukes in testing.

For the first time, we see SP1 actually distance itself from the RTM version. Microsoft’s claims of higher file operation speeds check out at least in this particular instance, where performance goes up by almost 24% from Vista RTM to Vista SP1.

Time and hardware constraints prevented us from running additional file copy and network copy tests, but other benchmarks we’ve seen around the Web suggest single-disk or disk-to-disk file operations are indeed quicker overall (but not always) in SP1. The picture for network copies is a little foggier, though, since one of our editors has run into performance issues with network shares on his own system following an update to SP1.