Personal computing discussed

Moderators: SecretSquirrel, notfred

 
MileageMayVary
Gerbil XP
Posts: 344
Joined: Thu Dec 10, 2015 9:18 am
Location: Baltimore

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 6:06 pm

Waco wrote:
just brew it! wrote:
MileageMayVary wrote:
The storage team at my work is setting up a 1 PB all flash NetApp cluster this year.

FWIW one of the managers at work mentioned today that one of our larger customers has a data "ingest rate" in excess of 1 PB per day. :o

(And no, the array isn't all flash in this case...)

One of my in-house designed systems averaged 1 PiB per day of real data for 3 weeks straight when a particular user unleashed hell on us. Singular user with a mission.

Another user who shall remain unnamed managed to nearly fill a pair of 40 PiB filesystems in a workday. Things move quickly at 1 TB/s+.

HOLYCRAP
Main rig: Ryzen 1600, R9 290@1100MHz, 16GB@2933MHz, 1080-1440-1080 Ultrasharps.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Topic Author
Posts: 2888
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 7:18 pm

MileageMayVary wrote:
HOLYCRAP

Big data means different things to different people. :P
Desktop: Z170A | 6700K @ 4.4 | 32 GB | Alphacool Eisblock Radeon VII | Heatkiller R3 | Samsung 4K 40" | 1 TB NVME + 2 TB SATA + LSI (128x8) RAID 0
NAS: 1950X | Designare EX | 32 GB ECC | 7x8 TB RAIDZ2 | 8x2 TB RAID10 | FreeNAS | ZFS | LSI SAS
 
just brew it!
Gold subscriber
Administrator
Posts: 52632
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 8:07 pm

Waco wrote:
MileageMayVary wrote:
HOLYCRAP

Big data means different things to different people. :P

Yeah, 80 PB in one day is pretty darned big by most people's standards!

Just take a moment to think about the kind of traffic and data capacity that the big Cloud service providers need to deal with, and you realize that there have to be an awful lot of datacenters with really fat pipes to the internet, and millions of SSDs and HDDs out there, all behind the scenes making our modern interconnected world function.

Furthermore, because of the crazy way copyright laws work, any service that provides "Cloud DVR" functionality (like Comcast and other cable providers do) must make a separate physical copy of all the digital bits for each user who records a program, in order to stay legal. Reading from a single shared copy to stream to multiple users, or using any kind of de-dup tech is not allowed. Essentially, they're relying on the same decades-old court rulings which carved out the copyright exception for time-shifting using home VCRs, but doing it "in the Cloud". So somewhere out there, scattered all around the world, there are entire datacenters dedicated to storing thousands of PB worth of redundant copies of TV programs...
Nostalgia isn't what it used to be.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Topic Author
Posts: 2888
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 9:14 pm

just brew it! wrote:
Furthermore, because of the crazy way copyright laws work, any service that provides "Cloud DVR" functionality (like Comcast and other cable providers do) must make a separate physical copy of all the digital bits for each user who records a program, in order to stay legal. Reading from a single shared copy to stream to multiple users, or using any kind of de-dup tech is not allowed. Essentially, they're relying on the same decades-old court rulings which carved out the copyright exception for time-shifting using home VCRs, but doing it "in the Cloud". So somewhere out there, scattered all around the world, there are entire datacenters dedicated to storing thousands of PB worth of redundant copies of TV programs...

Exhibit 1 for why lawyers and politicians should generally stay away from anything relating to technology (that they don't understand the implications of).

If only that could actually become true. :lol:
Desktop: Z170A | 6700K @ 4.4 | 32 GB | Alphacool Eisblock Radeon VII | Heatkiller R3 | Samsung 4K 40" | 1 TB NVME + 2 TB SATA + LSI (128x8) RAID 0
NAS: 1950X | Designare EX | 32 GB ECC | 7x8 TB RAIDZ2 | 8x2 TB RAID10 | FreeNAS | ZFS | LSI SAS
 
just brew it!
Gold subscriber
Administrator
Posts: 52632
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 9:30 pm

Waco wrote:
Exhibit 1 for why lawyers and politicians should generally stay away from anything relating to technology (that they don't understand the implications of).

If only that could actually become true. :lol:

Yup. The notion that -- since individual VCRs are explicitly a loophole -- they need to exactly simulate the functionality of a per-user VCR in a remote server somewhere in order to remain legal is just retarded. What a waste of equipment and electricity. I wonder how much of a factor "Cloud DVR" has been in keeping Seagate and Western Digital in business as consumer HDD sales have tanked... :lol:

I imagine as far as the cable operators are concerned it is still a net win versus providing in-home DVR boxes, since they don't need to deal with swapping out a physical DVR at someone's home when the HDD goes bad. They're just replacing a drive in a server in their datacenter.
Nostalgia isn't what it used to be.
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 27650
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 10:16 pm

Waco wrote:
Another user who shall remain unnamed managed to nearly fill a pair of 40 PiB filesystems in a workday. Things move quickly at 1 TB/s+.
Let me guess. Your network fabric is built on vibranium.
MileageMayVary wrote:
HOLYCRAP
Check his reported location and think about what's been going on there since 1943 or so.
Humans sleep soundly in their beds because rough cats stand ready in the night to visit violence on those who would do us harm.
 
MileageMayVary
Gerbil XP
Posts: 344
Joined: Thu Dec 10, 2015 9:18 am
Location: Baltimore

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 10:45 pm

Captain Ned wrote:
MileageMayVary wrote:
HOLYCRAP
Check his reported location and think about what's been going on there since 1943 or so.


Social Security Administration? I don't work for them.

I do love seeing the scale of things that people work on.
Main rig: Ryzen 1600, R9 290@1100MHz, 16GB@2933MHz, 1080-1440-1080 Ultrasharps.
 
just brew it!
Gold subscriber
Administrator
Posts: 52632
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 10:52 pm

MileageMayVary wrote:
Captain Ned wrote:
MileageMayVary wrote:
HOLYCRAP

Check his reported location and think about what's been going on there since 1943 or so.

Social Security Administration? I don't work for them.

SSA is older than that. Think "things that explode quickly" as opposed to "things that will implode slowly". :wink:
Nostalgia isn't what it used to be.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Topic Author
Posts: 2888
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Tue May 07, 2019 10:58 pm

just brew it! wrote:
MileageMayVary wrote:
Captain Ned wrote:

Check his reported location and think about what's been going on there since 1943 or so.

Social Security Administration? I don't work for them.

SSA is older than that. Think "things that explode quickly" as opposed to "things that will implode slowly". :wink:

The department of fast energy. :)
Desktop: Z170A | 6700K @ 4.4 | 32 GB | Alphacool Eisblock Radeon VII | Heatkiller R3 | Samsung 4K 40" | 1 TB NVME + 2 TB SATA + LSI (128x8) RAID 0
NAS: 1950X | Designare EX | 32 GB ECC | 7x8 TB RAIDZ2 | 8x2 TB RAID10 | FreeNAS | ZFS | LSI SAS
 
wizardz
Gerbil
Posts: 93
Joined: Thu Nov 02, 2006 12:58 pm
Location: Montreal, Canada

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Wed May 08, 2019 8:43 am

holy crap is indeed appropriate here..

i'm seeing PiB and TB/sec... please wait while i wipe the drool from my desk..
 
TheRazorsEdge
Gerbil Team Leader
Posts: 216
Joined: Tue Apr 03, 2007 1:10 pm

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Wed May 08, 2019 9:47 am

Captain Ned wrote:
Waco wrote:
Another user who shall remain unnamed managed to nearly fill a pair of 40 PiB filesystems in a workday. Things move quickly at 1 TB/s+.
Let me guess. Your network fabric is built on vibranium.


Infiniband and OPA can push that much data around with some prodding. You're looking at supercomputer-class hardware there... but that does make sense for an environment with at least two 40-PiB filesystems.

No need for unobtainium or equivalent.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Topic Author
Posts: 2888
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: Amusing bug in RHEL 7.4 - [UPDATED]

Wed May 08, 2019 9:50 am

Infiniband, generally. We have some OPA and custom interconnects (Cray), but the majority of the storage networking is IB.
Desktop: Z170A | 6700K @ 4.4 | 32 GB | Alphacool Eisblock Radeon VII | Heatkiller R3 | Samsung 4K 40" | 1 TB NVME + 2 TB SATA + LSI (128x8) RAID 0
NAS: 1950X | Designare EX | 32 GB ECC | 7x8 TB RAIDZ2 | 8x2 TB RAID10 | FreeNAS | ZFS | LSI SAS

Who is online

Users browsing this forum: No registered users and 2 guests