Page 1 of 7

Blackblaze's 2018-2019 HDD stats

Posted: Mon Jan 28, 2019 2:06 pm
by meerkt

Re: Blackblaze's 2018 HDD stats

Posted: Thu Mar 21, 2019 4:14 pm
by Topinio
Ugh. I just looked at this today, and came here to see if there'd been any discussion.

It's not pretty in light of HGST going away, that's for sure.

MFR Type AFR Ave. years deployed| :roll:
HGST Data centre 0.49% 2.83 :)
HGST Desktop 0.77% 4.55 :-?
HGST Enterprise 0.74% 0.64 :)
Seagate Desktop 2.44% 3.30 :(
Seagate Enterprise 1.17% 0.97 :-?
Toshiba | Surveillance | 0.99% 3.69 :-?
Toshiba | Enterprise 3.03% 0.25 :oops:
WDC Home NAS 3.96% | 3.91 :oops: :oops:


Bearing in mind that current **** 'spec' sheets list AFR as 0.44% for WD and Seagate, and 0.63% for Toshiba enterprise drives... :cry: :cry:

Re: Blackblaze's 2018 HDD stats

Posted: Thu Mar 21, 2019 9:04 pm
by Waco
Under 3% AFR is pretty decent for the workload BB puts on them and the models they're working with.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 7:06 am
by Topinio
Waco wrote:
Under 3% AFR is pretty decent for the workload BB puts on them and the models they're working with.

I'm not sure I could agree, Backblaze's usage doesn't seem to be so intensive that the drives in the enterprise / datacentre categories should have an excuse for their actual AFR to be 10 times their spec sheets' AFR's.

The Toshiba enterprise and WD NAS drives' AFR's are shocking at 3.03% and 3.96%, and the fact that the HGST disks are all under 0.8% and are 4 of the 5 best ones says a lot for their engineering.

HGST's only desktop drive included has an real AFR that's better than the other manufacturers' disks from the higher/pricier market segments, other than the ST10000NM0086 (0.45%) which is the only disk model that is in line with all the HGST products' reliability in this data set.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 8:05 am
by Chrispy_
WD will ruin HGST, mark my words.

WD themselves have seemingly lost all quality control and are now on my avoid list for mechanical drives. I have seen countless failed WD greens brought into me in the last three years and my own reason for ditching them at home (WD Reds) correlates with an increase in negative reviews and dropping egg ratings across their lineup.

Margins are getting tighter and WD's solution appears to have involved corner-cutting :(

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 11:46 am
by JustAnEngineer
Chrispy_ wrote:
WD will ruin HGST. WD themselves have seemingly lost all quality control. Margins are getting tighter and WD's solution appears to have involved corner-cutting :(
Obligatory: buggy whips.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 11:50 am
by Krogoth
HDDs are becoming the next "tape drive" in the 2020s. Solid-state media has pretty much taken over mainstream market while the masses opt for cloud storage for their bulk data needs.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 1:02 pm
by Waco
Topinio wrote:
I'm not sure I could agree, Backblaze's usage doesn't seem to be so intensive that the drives in the enterprise / datacentre categories should have an excuse for their actual AFR to be 10 times their spec sheets' AFR's.

I'm not exactly surprised that spec sheet AFRs are wildly optimistic. They're almost always far higher in practice.

Krogoth wrote:
HDDs are becoming the next "tape drive" in the 2020s. Solid-state media has pretty much taken over mainstream market while the masses opt for cloud storage for their bulk data needs.

Absolutely. They'll transition to being hard to write to next.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 1:15 pm
by Krogoth
Waco wrote:
Topinio wrote:
I'm not sure I could agree, Backblaze's usage doesn't seem to be so intensive that the drives in the enterprise / datacentre categories should have an excuse for their actual AFR to be 10 times their spec sheets' AFR's.

I'm not exactly surprised that spec sheet AFRs are wildly optimistic. They're almost always far higher in practice.

Krogoth wrote:
HDDs are becoming the next "tape drive" in the 2020s. Solid-state media has pretty much taken over mainstream market while the masses opt for cloud storage for their bulk data needs.

Absolutely. They'll transition to being hard to write to next.


They are already half-way there with the increasingly growing parity between I/O throughput and capacity of single disk. It takes over an entire day for a monstrous 14TiB+ HDD to fully write itself (which is kinda important for regenerating a RAID).

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 1:26 pm
by DancinJack
I'm fine with SATA SSDs taking over the mass storage market, but what I am not OK with is trading off traditional HDDs for QLC crap. An 8TB TLC SATA SSD for 250? Sign me up.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 1:27 pm
by Usacomp2k3
DancinJack wrote:
I'm fine with SATA SSDs taking over the mass storage market, but what I am not OK with is trading off traditional HDDs for QLC crap. An 8TB TLC SATA SSD for 250? Sign me up.

What if it was an 8TB QLC SSD for $250?

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 1:39 pm
by Waco
DancinJack wrote:
I'm fine with SATA SSDs taking over the mass storage market, but what I am not OK with is trading off traditional HDDs for QLC crap. An 8TB TLC SATA SSD for 250? Sign me up.

QLC is coming whether you like it or not.

TLC is coming whether you like it or not.

MLC is coming whether you like it or not.

...I think my brain just broke a little.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 5:44 pm
by Topinio
Chrispy_ wrote:
WD will ruin HGST, mark my words.

WD themselves have seemingly lost all quality control and are now on my avoid list for mechanical drives. I have seen countless failed WD greens brought into me in the last three years and my own reason for ditching them at home (WD Reds) correlates with an increase in negative reviews and dropping egg ratings across their lineup.

Margins are getting tighter and WD's solution appears to have involved corner-cutting :(

All true; sadly, we are now in the boat where Seagate's crappy drives are the best option available of the 3 on the market.

Krogoth wrote:
HDDs are becoming the next "tape drive" in the 2020s. Solid-state media has pretty much taken over mainstream market while the masses opt for cloud storage for their bulk data needs.

[...]

They are already half-way there with the increasingly growing parity between I/O throughput and capacity of single disk. It takes over an entire day for a monstrous 14TiB+ HDD to fully write itself (which is kinda important for regenerating a RAID).

SSDs are still technologically inferior to HDDs in a number of important ways, and in the primary economic metric of $/TB, but it might well be that HDDs cease to be economical to manufacture for use in consumer devices fairly soon. (The consumer ones are now spec'd to deliver 1/10 the data thoughput and 1/10 the UBER of the entry level enterprise ones, in order to be profitably both half the price and twice the capacity of a SSD.)

Thankfully, we're still seeing some innovations in the tech, multi-actuator should significantly improve IOPS and transfer bandwidth and it's been about a year since the announcement so hopefully not too long now.

Re: Blackblaze's 2018 HDD stats

Posted: Fri Mar 22, 2019 7:53 pm
by Krogoth
Topinio wrote:
Chrispy_ wrote:
WD will ruin HGST, mark my words.

WD themselves have seemingly lost all quality control and are now on my avoid list for mechanical drives. I have seen countless failed WD greens brought into me in the last three years and my own reason for ditching them at home (WD Reds) correlates with an increase in negative reviews and dropping egg ratings across their lineup.

Margins are getting tighter and WD's solution appears to have involved corner-cutting :(

All true; sadly, we are now in the boat where Seagate's crappy drives are the best option available of the 3 on the market.

Krogoth wrote:
HDDs are becoming the next "tape drive" in the 2020s. Solid-state media has pretty much taken over mainstream market while the masses opt for cloud storage for their bulk data needs.

[...]

They are already half-way there with the increasingly growing parity between I/O throughput and capacity of single disk. It takes over an entire day for a monstrous 14TiB+ HDD to fully write itself (which is kinda important for regenerating a RAID).

SSDs are still technologically inferior to HDDs in a number of important ways, and in the primary economic metric of $/TB, but it might well be that HDDs cease to be economical to manufacture for use in consumer devices fairly soon. (The consumer ones are now spec'd to deliver 1/10 the data thoughput and 1/10 the UBER of the entry level enterprise ones, in order to be profitably both half the price and twice the capacity of a SSD.)

Thankfully, we're still seeing some innovations in the tech, multi-actuator should significantly improve IOPS and transfer bandwidth and it's been about a year since the announcement so hopefully not too long now.


Sorry, HDDs have already lost the I/O throughput and random access game long ago. 2.5" 10K RPM units are now practically an endangered species. SLC media isn't that much more expensive and offer far superior performance. The only thing going for HDDs is being cheap per $$$/TiB which only matters for enterprise/SMB customers (who also happen to host cloud storage services for the masses). It is getting harder and harder for mainstream customer to justify a HDD over SSD in a normal desktop/laptop build.

Re: Blackblaze's 2018 HDD stats

Posted: Sat Mar 23, 2019 2:28 am
by Igor_Kavinski
Krogoth wrote:
It is getting harder and harder for mainstream customer to justify a HDD over SSD in a normal desktop/laptop build.


SSDs are pretty bad compared to HDDs as far as data reliability is concerned. Better be using a filesystem that handles and corrects silent data corruption. And backups are more important than ever now.

Another thing: never force a hard shutdown or reset on a busy SSD (and it might be even be busy internally if the hdd light isn't blinking). The SSD controller is always doing some household chores which have increased a lot in TLC and QLC SSDs. Hard shutdown has the risk of leaving them in a state from which the SSD controller may not be able to recover thus bricking the SSD for good. So if you value your data (or just don't want to deal with a downed SSD), invest in an enterprise SSD with data loss protection.

https://www.extremetech.com/computing/1 ... -your-ssds
https://betanews.com/2016/06/02/failed-ssd-data-loss/

Just searched to see if SSD data recovery is possible and found the following
https://www.ontrack.com/blog/2017/08/08 ... ate-drive/
https://www.ontrack.com/services/data-recovery/ssd/
https://www.minitool.com/data-recovery/ ... overy.html
https://www.stellarinfo.com/blog/recove ... ndows-mac/
https://www.gillware.com/data-recovery- ... ite-paper/:

Re: Blackblaze's 2018 HDD stats

Posted: Sat Mar 23, 2019 5:35 am
by Topinio
Krogoth wrote:
Sorry, HDDs have already lost the I/O throughput and random access game long ago.

No ****, Sherlock.

But that's like the GHz game: clearly important, but not the only aspect that matters.

Krogoth wrote:
The only thing going for HDDs is being cheap per $$$/TiB

Yes, and that's what a lot of people care about, see the GHz race.

There's also absolute capacity available -- for desktop client devices (built with consumer-tier parts), a 14 TB HDD is an option and a 14 TB SSD simply is not.

(In the latptop sphere, that's not really true currently since the launch of 4 TB consumer SSD's last year rendered mostly-irrelevant the 5 TB 2.5" HDD's.)

Hard drives are also still winning on the shouldn't-be-a-factor-but-sadly-is metric of recoverability, and were winning on longevity in deployment but I am not convinced that the current HDD's will continue this when we look back at them.

Krogoth wrote:
which only matters for enterprise/SMB customers

Nah, nope, not so. There's a whole market segment that's still hugely significant (though shrinking) of consumers -- and PHB's / employees, actually -- who want that cheap capacity despite the slowness.

The peddlers of crappy computers know their business and know full-well that the £270 machine with 1 TB, or the £320 one with 2 TB, is what a lot of people buy.

Proof: 97% of desktops and 43% of laptops listed at Argos today have a HDD rather than an SSD.

Krogoth wrote:
which only matters for enterprise/SMB customers (who also happen to host cloud storage services for the masses).

Lol, I'm assuming that 's an amusing thinko and you don't actually mean you think that for the HDD companies these are remotely the same customer segments or the same product ranges. LMK if otherwise.

Krogoth wrote:
It is getting harder and harder for mainstream customer to justify a HDD over SSD in a normal desktop/laptop build.

Nah, that might be true in certain bubbles, but IME Igor_Kavinski's first sentence reflects the typical view among more knowledgable consumers, and the less knowledgable want cheap first, then TB -- it's kinda replacing GHz as the new single-factor variable they look at.


But none of this is the point, really. Hard drives are getting crappier, and that sucks. New technological advances are still happening, and that's something to look forward to.

I'm not sure if the race to the bottom will allow it, but with dual-actuator consumer HDD's would become capable of saturating the 6 Gbps SATA link when reading or writing that large sequential e.g. video file, which is a good thing, as is the doubling of their sucky IOPS given that people are still buying the damn things.

Yes, they're being developed because of the enterprise array scenario (and, I'd imagine, for cloud storage people) -- where dual-actuator could provide for either a quantum leap in IOPS/TB (fixed spindles and capacity), a doubled capacity (fixed IOPS/TB and spindles), or a half-sized solution with the same capacity and IOPS/TB of the current possibility -- but I'd hope to see it trickle down to desktops next year.

Re: Blackblaze's 2018 HDD stats

Posted: Sat Mar 23, 2019 9:13 am
by Krogoth
Igor_Kavinski wrote:
Krogoth wrote:
It is getting harder and harder for mainstream customer to justify a HDD over SSD in a normal desktop/laptop build.


SSDs are pretty bad compared to HDDs as far as data reliability is concerned. Better be using a filesystem that handles and corrects silent data corruption. And backups are more important than ever now.

Another thing: never force a hard shutdown or reset on a busy SSD (and it might be even be busy internally if the hdd light isn't blinking). The SSD controller is always doing some household chores which have increased a lot in TLC and QLC SSDs. Hard shutdown has the risk of leaving them in a state from which the SSD controller may not be able to recover thus bricking the SSD for good. So if you value your data (or just don't want to deal with a downed SSD), invest in an enterprise SSD with data loss protection.

https://www.extremetech.com/computing/1 ... -your-ssds
https://betanews.com/2016/06/02/failed-ssd-data-loss/

Just searched to see if SSD data recovery is possible and found the following
https://www.ontrack.com/blog/2017/08/08 ... ate-drive/
https://www.ontrack.com/services/data-recovery/ssd/
https://www.minitool.com/data-recovery/ ... overy.html
https://www.stellarinfo.com/blog/recove ... ndows-mac/
https://www.gillware.com/data-recovery- ... ite-paper/:


>Implying that newest batch of HDDs aren't next to impossible to perform disaster recovery due to platters being too dense and/or rely on exotic technologies (Helium, HAMR/MAMR) that make "clean room" recovery an impossibility.

Multi--actuator HDDs aren't even new. It is just another last grasp on trying to make HDDs relevant in light of heated competition against solid-state media. The reason why it wasn't used beyond prototype/experimental units in the 1970s-1980s was because they are unreliable and prone to failure (more points of it). The same reason why PMR stayed in the experimental tech box until LMR was hitting the superparamagnetic wall back in the early to mid 2000s. It is funny now that PMR is also hitting the same superparamagnetic wall.

There's a reason why every existent HDD manufacturer has a solid-state media line-up now. They have already seen the writing on the wall even battle-harden veterans like Seagate. They have a very good idea on who is buying up the bulk of HDD media these days. (Here's a hint, it isn't coming from retail/etailers)

HDDs are becoming the next "Tape Drive/Archival optical media drive" in the 2020-early 2030s.

Re: Blackblaze's 2018 HDD stats

Posted: Sat Mar 23, 2019 12:40 pm
by derFunkenstein
Waco wrote:
Topinio wrote:
I'm not sure I could agree, Backblaze's usage doesn't seem to be so intensive that the drives in the enterprise / datacentre categories should have an excuse for their actual AFR to be 10 times their spec sheets' AFR's.

I'm not exactly surprised that spec sheet AFRs are wildly optimistic. They're almost always far higher in practice.

Some of that's gotta be a difference between what the manufacturer's test environment and a real production environment looks like, right?

Re: Blackblaze's 2018 HDD stats

Posted: Sun Mar 24, 2019 5:04 am
by Topinio
derFunkenstein wrote:
Waco wrote:
Topinio wrote:
I'm not sure I could agree, Backblaze's usage doesn't seem to be so intensive that the drives in the enterprise / datacentre categories should have an excuse for their actual AFR to be 10 times their spec sheets' AFR's.

I'm not exactly surprised that spec sheet AFRs are wildly optimistic. They're almost always far higher in practice.

Some of that's gotta be a difference between what the manufacturer's test environment and a real production environment looks like, right?

Well, that just means that the test simulation environment is very very poor.

Obviously it being an order of magnitude out in the favourable direction is just a minor whoopsie due to how hard the problem is to simulate, right folks?

Edit to add: I think this is probably why the TBW of consumer drives has dropped to 1/10th of that of the enterprise ones -- they know.

Re: Blackblaze's 2018 HDD stats

Posted: Sun Mar 24, 2019 1:20 pm
by derFunkenstein
Topinio wrote:
Well, that just means that the test simulation environment is very very poor.

Obviously it being an order of magnitude out in the favourable direction is just a minor whoopsie due to how hard the problem is to simulate, right folks?

Edit to add: I think this is probably why the TBW of consumer drives has dropped to 1/10th of that of the enterprise ones -- they know.

I'm positive that nobody but Backblaze has a production environment quite like that. Backblaze's numbers are worst-case, to me, and all of that doesn't matter anyway because I'm buying a single drive. If mine dies, it dies and I'm out of luck. Hope I had a backup. :lol:

Re: Blackblaze's 2018 HDD stats

Posted: Sun Mar 24, 2019 11:46 pm
by Waco
derFunkenstein wrote:
I'm positive that nobody but Backblaze has a production environment quite like that. Backblaze's numbers are worst-case, to me, and all of that doesn't matter anyway because I'm buying a single drive. If mine dies, it dies and I'm out of luck. Hope I had a backup. :lol:

My environments are quite similar to Backblaze in practice. Many others are as well.

Re: Blackblaze's 2018 HDD stats

Posted: Mon Mar 25, 2019 6:57 pm
by Topinio
derFunkenstein wrote:
Backblaze's numbers are worst-case, to me, and all of that doesn't matter anyway because I'm buying a single drive. If mine dies, it dies and I'm out of luck. Hope I had a backup. :lol:

On that first point, I rather doubt that Backblaze is even near worst case, their environment is likely reasonably well-controlled and the pod-fill algorithm combined with the reasonably low data churn might well result in lower average stress per drive than many other environments IRL.

On the second point, that attitude is probably widely shared and a contributing factor in the manufacturers' dive off the quality cliff.

To which, I should ask: are your backups on a recent-quality mechanical drive? :wink:

Re: Blackblaze's 2018 HDD stats

Posted: Mon Mar 25, 2019 6:59 pm
by derFunkenstein
OK you guys probably know better than I about production environments, particularly Waco.

My backups are on an HGST 6TB drive I bought in November. It's unplugged and turned off except for when I'm backing up. It's on-site still, so that's not great, but it's as good as I can do without "cloud" backups.

Re: Blackblaze's 2018 HDD stats

Posted: Mon Mar 25, 2019 7:41 pm
by Usacomp2k3
My backup strategy to my single-drive NAS is: 2nd hot internal drive plugged into the NaS that does nightly mirrors. 3rd external USB enclosure that does nightly mirrors. I disconnect this one and put it in the safe whenever we leave the house for >24 hours. Also critical data is cloud backed up to OneDrive. All data is cloud backed up via BackBlaze.

Re: Blackblaze's 2018 HDD stats

Posted: Tue Mar 26, 2019 9:19 pm
by meerkt
Another sad aspect is how Seagate and WDC decided to transform mainstream into 5400 rpm, instead of the 7200 rpm it used to be. And that's compounded when the average capacities and data sizes continue to grow.

At least Toshiba doesn't play this game.

Re: Blackblaze's 2018 HDD stats

Posted: Tue Mar 26, 2019 9:36 pm
by Redocbew
I would guess that bare drives for local mass storage is a shrinking market these days. Most of them are probably going into external USB enclosures, or home NAS appliances where the difference between 5400 and 7200 RPM isn't a real dealbreaker, but I could be wrong.

Re: Blackblaze's 2018 HDD stats

Posted: Wed Mar 27, 2019 6:58 am
by Chrispy_
meerkt wrote:
Another sad aspect is how Seagate and WDC decided to transform mainstream into 5400 rpm, instead of the 7200 rpm it used to be. And that's compounded when the average capacities and data sizes continue to grow.

At least Toshiba doesn't play this game.


I, for one, am pleased that 'mainstream' speeds have dropped to 5400rpm. Mechanical storage is mainly used for media when talking about consumers and that doesn't need vast speeds. What it *does* need to be, for consumers, is quiet, reliable, vibration-free, and energy efficient.

An obnoxious drive that gets hot, uses more power and wears through its bearings faster than necessary is not in the interest of consumers, especially since anything a consumer needs to do that demands performance should be done on an SSD these days. Mechanical drives are for bulk storage of media content and backup, neither of which need blistering performance.

I have 4TB Ironwolf drives (5900rpm) and 8TB HGST drives (7200rpm) and the difference in sequential speeds is 150MB/s vs 180MB/s. Both are attrocious at mixed IO, dropping down to double-digit transfer rates, and 4K random performance is nonexistent, at under 1MB/s for either drive. Anyone wanting "performance" from a mechanical drive is quite simply doing it wrong. Not even multi-drive hardware-based RAID0 arrays are enough to rescue mechanical drive performance; They are truly a bulk storage tool, unable to compete in random workloads with even with eMMC at the dirt-cheap, almost disposable end of the market.

Re: Blackblaze's 2018 HDD stats

Posted: Wed Mar 27, 2019 8:34 am
by meerkt
Is there any data showing 5400rpmers are more reliable than 7200?

The main issue is linear speed, nor random I/O. In the old days reading a whole HDD took a few minutes. Now it's a few minutes per large file, and half a day for a whole HDD. An extra 33% (5400rpm) is not that much but it's far better than nothing.

Noise or heat weren't too bad in any of the 7200rpmers I've had. Quiet is nice, and the solution is to spin down most of the time, even 5400rpmers. Power isn't a concern unless you're a datacenter.

Re: Blackblaze's 2018 HDD stats

Posted: Wed Mar 27, 2019 9:28 am
by Waco
meerkt wrote:
Is there any data showing 5400rpmers are more reliable than 7200?

In the long run, yes. Bearings and motors last much longer at the lower RPM.

I'd happily welcome lower-RPM drives for bulk storage but there's a minimum speed needed to keep the head flying properly at the inside edge of the platters (especially relevant for helium drives).

Re: Blackblaze's 2018 HDD stats

Posted: Wed Mar 27, 2019 9:47 am
by meerkt
Problems might manifest much later than the expected service life, no? If a 7200 drive only lasts 100K hours instead of 130K (well, <5400rpm) that's not a big loss. And if it's spun down most of the time it's completely irrelevant.

I'm not saying they shouldn't produce 5400rpm, but 7200rpm shouldn't be considered a relative luxury item.