TR Forums

Run a full disk check (chkdsk driveletter: /f /v /r) on an external hard disk (USB or eSATA), and all the memory you've got be consumed and the system responsiveness grind to a halt

I've seen this bug enough times on various machines running the 64-bit version. I don't whether the 32-bit version is affected.

This was reported since beta (or RC?) and Microsoft's response has been that it's not a bug.
Why it is intended to work this way, I have no idea.

<flabbergasted>

Doesn't do it in XP or Vista, but does in Win7...

- edit -

I've been reading up on this, and Sinofsky reckons it speeds up the chkdsk process, using up all the memory. Quite a few people responded saying it doesn't make any difference. I haven't timed it myself, but it doesn't appear to be any faster than predecessors. The amusing thing is, chkdsk sometimes asks whether you want to restart and run the check during startup, so what's this new scenario, "do you want to run it in single-process mode, or in single-process mode?"?

I think I'll ask for someone to send me chkdsk.exe from Vista x64 and give that a try. I have to regularly do disk checks on customers' hard disks on my machine, and I don't have a quad-core processor with 4GB of RAM so I can watch chkdsk!

http://blogs.msdn.com/b/e7/archive/2009 ... eport.aspx

It's tied to the /r switch, which finds bad sectors and attempts to recover information.

The assumption is that such a process is important to you and as such it is granted broad resource access to acheive the job.

Edit: Yes, this is a purposeful change to chkdsk in 7.

Ryu Connor wrote:

http://blogs.msdn.com/b/e7/archive/2009/08/10/what-we-do-with-a-bug-report.aspx

It's tied to the /r switch, which finds bad sectors and attempts to recover information.

The assumption is that such a process is important to you and as such it is granted broad resource access to acheive the job.

Edit: Yes, this is a purposeful change to chkdsk in 7.

On the one hand, yeah if you've got a disk with sectors that are going bad, you've likely got bigger things to worry about than whether your system remains optimally responsive during the recovery effort.

On the other hand, modern systems have enough CPU horsepower and RAM that it really should be possible to design a tool like this that can operate without dragging system performance down the toilet. The speed of the recovery operation is ultimately limited by the speed of the hard drive and/or USB interface; there's no excuse for chewing up humongous amounts of CPU and RAM on a recent model PC to do something that really ought to be a cakewalk for almost any PC built in the past decade.

The behavior may very well be "by design"; but if so, it is a pretty crappy design.

mikeymike wrote:

The amusing thing is, chkdsk sometimes asks whether you want to restart and run the check during startup

That generally applies to the system disk, but may happen under circumstances where there are files locked in use on non-system disks.

JBI wrote:

The behavior may very well be "by design"; but if so, it is a pretty crappy design.

This change must have been debated by the file system team. There's no way the whole team was in lockstep to the idea originally. Someone must have no doubt voiced the concept that the perception of the feature would get them mocked as well.

That the whole team is idiots isn't a valid argument. Nor does one have to agree that their choice was 100% correct, but it seems fair to assume that their choice was guided by a better understanding of the file system, tool, and the needs of their user base than we have.

I think Sinofsky is deflecting the issue's priority by claiming "it's not a bug, it's a feature". I bet it'll be fixed in SP1.

If it had any performance advantage whatsoever, that would have been a central point to the discussion in the team. Performance analyses would have been generated, he would have the results to prove it, and they could be independently verified.

As it is, IMO, someone screwed up while updating chkdsk for Win7, or Win7 has some flaw that shows when chkdsk is running like this. My bet is the latter. The flaw isn't a particularly high priority (but could be quite serious) to fix because it has only become obvious in one scenario, and it would take more time to fix than they can spend, considering the priority of the issue.

I've just run a full chkdsk on a drive on Win7 64-bit, which took 50 minutes (I used the 'timethis' program to get an exact reading). I'll do the same with my very old XP laptop and see if it gives a similar reading. The Win7 machine has everything going for it, so it should trounce the XP machine at this job, if Sinosky's claim has any merit.

If someone has Vista x64, if you wouldn't mind, please contact me to arrange the sending of chkdsk.exe.

mikeymike wrote:

I bet it'll be fixed in SP1.

I wouldn't. When they announce it is by design, they mean it.

You're applying a great deal of logic to the item that is irrelevant. Even if it is slower... so what? They've already deemed the behavior as by design. The behavior has existed and been reported since before RTM, the press has been all over it, and the behavior has seen gross mischaracterization (even TR reported on it: http://techreport.com/discussions.x/17364 ). That hasn't pressured them to change the behavior, create a KB article about it, or release a patch. They've stood by their by design statement from months ago.

Do you think your testing is gonna be the one man to move the mountain? To show them that their viewpoint is all wrong?

Not liking a behavior does not make it a bug.

Edit: http://www.zdnet.com/blog/bott/a-killer ... ry-no/1235

axeman wrote:

It could also be a "none of us is as dumb as all of us" scenario.

Could be, but the scope of these examples on different ends of the criticality scale.

A cli tool that interacts with the file system directly is a whole lot different than a UI decision/fight.

The level of outrage is totally different as well.

How many people say they won't leave XP because of chkdsk?
How many people say they won't leave XP because of the UI?

I also don't find that blog post particularly damning or even a good example of how the process was somehow ruined. He didn't like it and he apparently didn't like having to compromise against what the other teams thought, but he needs to get over it. If anything this only reinforces my point. If there's that much contention during the design you don't think somebody didn't raise a red flag over this high memory usage? If his example is indeed the norm (something he couldn't confirm) then the change saw plenty of debate and the file system team apparently won.

I just re-read my posts and I don't see where I've said anything that implies "we aren't smart enough to question or understand the rationale" or that "none of us is as dumb as all of us" is impossible.

The most I said is that it "seems fair to assume" that the file system team had more facts about the situation than us ("better understanding of the file system, tool, and needs of their user base"). You of course don't have to give the file system team the benefit of the doubt. You can assume that you know more than them and that they are all idiots if you want, that's your perogative. All I know for sure is that I am an end user, I simply employ said technologies.

I've never seen the user feedback they receive about NTFS or chkdsk. 50% of their feedback might say that chkdsk should draw pictures of flowers while working. IT workers are sensitive and really in tune with their feelings so that being true would hardly shock me. I also have never done any programming interaction with or for NTFS (a reality I suspect for most if not all of this forum). So hey, if someone from the incredible edible internet wants to roll into here and issue forth from the fountain of knowledge, I'm all eyes. Be warned though, I am armed with a large amount of grains of salt. The best I could find on a short Google was the link I provided to Ed Bott who also says it's not a bug and details how it completed a chkdsk of his system in fifteen minutes.

Nothing unveiled has reavealed the file system team to be idiots. Since this isn't a bug, we have disagreement on implementation. There are lots of those in the world. For instance, I really like UAC. I suspect I'm outnumbered by the people who don't. Perhaps one day the rest of the world will realize I'm right. I expect I might get named king of the earth on that day too.

You posted your blog link as a "none of us is as dumb as all of us" example. What I see when I read that is a story of conflict over a choice during development. I'd point out that I had already assumed just such a fight had occured. "There's no way the whole team was in lockstep to the idea originally. Someone must have no doubt voiced the concept that the perception of the feature would get them mocked as well." Your link details just how hard slog it is to get a feature implemented. With the caveat of course that said blogger isn't even sure if his situation was representative of the norm. So I voiced my disagreement that your link was a good example for "none of us is as dumb as all of us", but that's not the same as saying that "none of us is as dumb as all of us" is impossible. Nor is it saying that "we aren't smart enough to question or understand the rationale". I am on the other hand saying that I think your link reinforces my point that this behavior in chkdsk is in fact by design.

This specific issue made huge waves in the media. It was on all the tech news sites with the various pundits claiming Win7 was doomed to miss RTM. It garnered specific posts about the process of bug examination and the process of that examination of this direct issue by MS. So not only did this specific item receive scrutiny before release, but it continued to attract attention all the way into ship. The answer the entire time that process occured was, "By Design."

Meanwhile remember Vista having trouble with network and audio?

This: http://www.zdnet.com/blog/hardware/micr ... -issue/724

Remember all the press that got. Did you see it never get patched?

It got press, it got a blog post by Mark Russinovich detailing it in depth, and it got a patch.

One of these things is not like the other. Notice how this chkdsk thing got the press, got the blog post saying it's not broke, and then didn't get a patch.

Again I never said "we aren't smart enough to question or understand the rationale" nor did I say that "none of us is as dumb as all of us" isn't possible.

Then what is my point? My point is this isn't a bug. My failing was apparently not getting to that point in a quicker manner, but I figured a lack of reasoning would invite conflict. Apparently a plethora of reasoning (and direct references) still invites conflict.

Who knew?

List of SP1 changes.

The behavior of chkdsk /r is not listed as a change.

The only chkdsk related change is an actual bug that applies to Vista too. Albeit technically it's not a chkdsk change, it's a volsnap.sys change.

What did I win from the bet?

Ryu Connor wrote:

What did I win from the bet?

A lump of coal in your stocking Perhaps they'll patch it but not publicly document the patch

I still find it all but impossible to imagine a scenario where it is necessary (or even desirable) to drag performance of the entire system down the tubes to perform this sort of operation. I could maybe understand it if the drive being diagnosed was the system drive, since you'd need to take pains not to step on (or get confused by) other activities being performed by the OS. But that's not the case here.

The chkdsk tool should be spending the vast majority of its time just sitting there waiting for I/O to complete. While in this state it should be consuming minimal system resources other than whatever RAM is required to hold state information required by the metadata consistency check. Furthermore the metadata gets checked regardless of whether the /r switch is specified, so /r really should not be a factor in RAM usage. Scanning a drive for bad sectors should be one of the cheapest -- in terms of CPU and RAM usage -- disk diagnostics you can perform, since it is essentially all I/O (which is handled via DMA with minimal CPU intervention), and you don't need to cache the data you're reading.

I've done OS work. I've written disk diagnostics. I stand by my claim that this sounds like crappy software design -- if not in the chkdsk tool itself, then in some critical OS system call(s) that chkdsk relies on.

I've done OS work. I've written disk diagnostics. I stand by my claim that this sounds like crappy software design -- if not in the chkdsk tool itself, then in some critical OS system call(s) that chkdsk relies on.

So you're not sure why either.

I also don't see in your thought processes what you might do to speed your hypothetical chkdsk up.

I don't want to hear about how you dislike the feature. I don't want to hear they're idiots. I want to know why it uses that much memory. Now you can tell me afterwards you wouldn't have taken that path, but I want to know why they took that path first.

Steven Sinofsky wrote:

In this case, we haven’t reproduced the crash…. [T]he design was to use more memory on purpose to speed things up, but never unbounded — we requset [sic] the available memory and operate within that leaving at least 50M of physical memory. Our assumption was that using /r means your disk is such that you would prefer to get the repair done and over with rather than keep working.

So, JBI. Your hypothetical chkdsk has access to all of the physical memory except the last 50MB and it must be made to run as a fast as possible.

Tell me why that would speed up your tool. Then tell me an alternate way to do the same thing - the way you'd have chosen.

Edit: Apparently 2008 R2 has the same behavior.

mr. ryu conner, why on earth do you feel it necessary to contradict the original poster and his feelings that chkdsk in win7 is a horrible experience? you continue to rant on and on about how it is designed that way. what does that matter if the OP is having an issue? the OP was complaining about the time he has to sit there and stare at his pc while it bogs down to a crawl doing a chhkdsk. then you come along and basically say, "stop complaining it is designed that way." just sit there and take it which is a pathetic attitude. if you don't like something, by all means speak up! you seem to want people to just shut up and stop complaining. well nothing is perfect, and that includes the win7 chkdsk tool--no matter how or why it was designed it is still too damn slow! don't piss on me and then tell me it's raining...

Can someone else running Win7 SP1 confirm that this issue has been fixed? I've got two USB hard disks formatted as NTFS having chkdsk (/f /v /r) running at the same time and memory usage isn't climbing.

Admittedly both the disks are freshly formatted, I'm just testing them, but I'm hoping that this isn't an isolated incident

Ryu Connor wrote:

JBI wrote:

The behavior may very well be "by design"; but if so, it is a pretty crappy design.

This change must have been debated by the file system team. There's no way the whole team was in lockstep to the idea originally. Someone must have no doubt voiced the concept that the perception of the feature would get them mocked as well.

That the whole team is idiots isn't a valid argument. Nor does one have to agree that their choice was 100% correct, but it seems fair to assume that their choice was guided by a better understanding of the file system, tool, and the needs of their user base than we have.

Man, he just called you a tool! You gonna' take that?

The "bug" never always triggered to begin with.

http://www.zdnet.com/blog/bott/a-killer-windows-7-bug-sorry-no/1235

I would get this bug when I try to enable ahci with an SSD on my AMD HTPC. Using a OCZ SSD on a win7, 790gx i will get chkdsk error but I don't have the problem when I enable it on the 1366 intel machine. I just have leave the driver off on the AMD machine.

Oh well, not fixed.

May 2, 2011 -

whether it was "by design" or not it's alarming behavior when chkdsk uses 23.5 GIGABYTES of RAM, leaving me 500MB to work with. Sure that's plenty to surf the web and whatnot but that's not why I have a 980x and 24GB of RAM.

If it's supposed to be using nearly 24GB of RAM .... well okay. But I'm running it on a (relatively) fast Spinpoint F3 (1TB) that's got 64% free space. Coming up on an hour with no end in sight... (10% complete of Stage 4)

Win 7, for all the good things about it there are some really aggravating "features" about this OS.
23.5G of RAM to run chkdsk is one.
No Folder Size column is another... (sorry, I just like to vent about that one).

To Whom, et al,

Just in case anyone hasn't "solved" this feature, using the 32 bit version of chkdsk found here,
C:\Windows\winsxs\x86_microsoft-windows-chkdsk_31bf3856ad364e35_6.1.7600.16385_none_c1bcb003ee041301
will automatically limit the memory usage to 3,387,460 KB. Yeah, that 32 bit 'bug"/Feature of 32 bit addressing
only 4gb of ram. How nice! *wink* Thanks Bill Gate!

Windows 8 "fixed" this by making the behavior require a switch (/perf)

http://www.eightforums.com/tutorials/62 ... s-8-a.html

[quote="Ryu Connor"]Windows 8 "fixed" this by making the behavior require a switch (/perf)


Ironically, I'm running 7 because 8 hosed one of my partitions and I'm trying to recover
8 from file corruption. I'm running chkdsk from 7 to identify bad sectors of which it found 3
from the last run. I tried running chkdsk from within 8 and it didn't find those bad sectors.
I'll look at the link you posted and see if I can get 8's chkdsk to run in startup at will...