RAM Disk usage + Handbrake = Awesome

[mmmmmdonuts21]

So I recently upgraded to 16gb of RAM for my system on black Friday. I know most people don't think this is necessary and I agree with you but I have been fiddling around with RAM disk as a scratch disk for photos and handbrake and it is fantastic. (This is what I am using http://members.fortunecity.com/ramdisk/RAMDisk/ramdiskent.htm)

Handbrake is where this really, really shines. I do a lot of bluray encoding (on an X6 1055T @3.75ghz) and I have seen a huge progression in encoding times from mechanical HDD to SSD to Ram Disk.

2 Hour BluRay (Iron Man 2)

Mechanical HDD (1TB WD Black) FPS avg: (18 peak, 10 avg) - completes in about 4-4.5 hours
SSD (kingston 128gb (I know its not the fastest)) - FPS avg: (25 peak, 17 avg) - completes in about 2.5-3 hours
RAM Disk - FPS avg: (43 peak, 32 avg) - completes in about 1.5 hours

I can't believe how much of a difference this makes. It is close to three times faster than a mechanical HDD and I am sure if I could put the entire file on the RAM disk rather than just running handbrake in the RAM disk and having it write the file to the SSD I am sure I could get rid of another bottleneck.

I am actually going to try it with a DVD and see how fast I can rip through with the whole file being on the RAM disk. I am hoping something like ten minutes.

[Shining Arcanine]

I have 8GB of RAM and I use it as a tmpfs for compiling my operating system. A tmpfs is a dynanmically sized RAM disk, so it only requires space for what you store and no more.

I recently enabled a feature in my system's package manager that adds debug data to the mix so that I could file better bug reports when things go wrong (ironically, nothing has went wrong). That increased the compilation space requirements by about a factor of two. Prior to that change, I could get by with 8GB of RAM with a 8GB swap file so that the virtual memory system can handle peak parts of the compilation that needed more than 8GB of space, specifically when open office is being compiled alongside many other things. Now that is no longer enough.

I can see how people would need 16GB of RAM. Personally, I would like a system with 32GB. The extra headroom would mean that I could do tomorrow's compilations in RAM with today's hardware.

[Flying Fox]

Is it just running Handbrake on the RAM disk? Do you also have to configure Handbrake to use the RAM disk as its scratch drive and such? I am a bit of a noob in that area so if you can show where in the settings to change that will be great.

There are lots of other uses of the large RAM. Running VMs is a prime example. My work system has 12GiB and 3 VMs later I am already low.

[mmmmmdonuts21]

Is it just running Handbrake on the RAM disk? Do you also have to configure Handbrake to use the RAM disk as its scratch drive and such? I am a bit of a noob in that area so if you can show where in the settings to change that will be great.

Yes and no. Basically I downloaded handbrake to install it on the RAM disk (which for me is drive B: ) instead of installing it in the original program files directory. From there I would go in and manually open up the handbrake application. Once in handbrake I take the file from its original location (in my case my windows desktop) and then output the file to the same location. I hope that makes sense.

I am curious on how other people do with this with less cores and with a mechanical drive instead of a ssd as backup. I am most likely going to try and test that out next. Most of the time DVD's can't get my cpu to 100% on all cores anymore since doing this. Blu Ray's still do though.

Where you have to be careful is once you turn off your computer you lose everything, so you have to make sure you export your RAM disk drive image before you shut down and then import the image when your computer boots back up.

[kamikaziechameleon]

Wow this is simply amazing. I stumbled in here not knowing anything about anything and this is just cool. I'm thinking of digitizing my limited blu-ray collection, can you provide an abreviate walkthrough with ram disk and handbrake and what not?

[wibeasley]

Where you have to be careful is once you turn off your computer you lose everything, so you have to make sure you export your RAM disk drive image before you shut down and then import the image when your computer boots back up.

You do this manually every time, or does RAMdisk provide customizable scripts (or something) so it's automated?

[ChronoReverse]

You must be using the pretty low level quality settings assuming you're encoding to h.264. It's way too easy to become CPU limited when encoding assuming you're not using so much bitrate that it negates the purpose of using h.264.

[mmmmmdonuts21]

You must be using the pretty low level quality settings assuming you're encoding to h.264. It's way too easy to become CPU limited when encoding assuming you're not using so much bitrate that it negates the purpose of using h.264.

I am just using the default encoding settings to (RF of 20) etc. and encoding into a mkv file and using AC3 and DTS passthrough. I do have six cores @ 3.75 so with DVD's it fluctuates depending on the quality and length of the DVD if all cores are 100%. When I said they are not 100% I mean a core or two could dip into the 85-95% range briefly every once in a while.

You do this manually every time, or does RAMdisk provide customizable scripts (or something) so it's automated?

The program I am using (Qsoft Ramdisk) has this functionally to do it automatically on shut down of your computer and load it back up when you boot your computer back up. I am almost positive you can have it do it every few minutes with script of some sort but I haven't tested it yet because I really don't shut off my computer that often because I am running BOINC. (and its connected to a UPS)

Wow this is simply amazing. I stumbled in here not knowing anything about anything and this is just cool. I'm thinking of digitizing my limited blu-ray collection, can you provide an abreviate walkthrough with ram disk and handbrake and what not?

I can try and write something up quickly this weekend, but at the moment I don't have the time.

I hopefully hope to test it on my laptop to gain a sense of how a slower system will handle this.

[mmmmmdonuts21]

So I decided to give this a try on my laptop to see what effects a RAM disk would have on a slower system. Basically it has little to no effect on encoding times, but I am guessing its because of a cpu bottleneck rather than a HDD bottleneck. If you have a quad core system (or hex core) perhaps your results will be different and benefit from this.

Really I believe it is a game of exploiting the Read/Write transfer rates of a HDD, and then hopefully having enough CPU power to maximize the results. The reason I looked into exploiting this in the first place was because of the huge encoding time decrease from going from mechanical HDD to SSD, so I wanted to go one step further and go SSD to RAM.

Unfortunately, I don't really have anymore systems I can really test it on at the moment so I would really be curious if other people see faster encoding times by using a RAM Disk. I believe the greater the bitrate/resolution (1080p as opposed 480p) the greater you have a chance of seeing benefits possibly.

Bottom line: It may help you achieve faster encoding times but like anything it depends largely on the environment its put in (and therefore where your bottlenecks occur).

Laptop Specs - in case you were wondering
i3@2.13GHz
4gb Ram
768 mb Ram Disk
500 WD Scorpio Blue

[Zoomastigophora]

It doesn't make sense to me that putting the Handbrake application on the RAM disk would speed things up unless Handbrake uses a scratch area. Once a program starts, it will run almost entirely in memory, assuming you have enough RAM, so putting Handbrake on the RAM disk should only speed up its load time. What you should test is the following:

• Handbrake on hard drive; source and destination file on the same hard drive
• Handbrake on hard drive; source on SSD, destination on another hard drive (or vice versa)
• Handbrake on hard drive; source on RAM disk, destination on SSD
• Handbrake on RAM disk; source on RAM disk, destination on SSD

The idea is that encoding is both IO and CPU bound. Generally, you don't want to encode a file from the same disk that you're outputting to, so theoretically, the best performance you should get is when the source is entirely on the RAM disk and you're outputting to the SSD. It shouldn't matter whether Handbrake is loaded from the RAM disk or from a hard drive.

[Shining Arcanine]

The idea is that encoding is both IO and CPU bound. Generally, you don't want to encode a file from the same disk that you're outputting to, so theoretically, the best performance you should get is when the source is entirely on the RAM disk and you're outputting to the SSD. It shouldn't matter whether Handbrake is loaded from the RAM disk or from a hard drive.

Something cannot be both I/O bound and CPU bound. It has to be one or the other. Being bound by something means that you cannot improve performance without improving that thing.

[JustAnEngineer]

There are values between zero and one for computing performance as for nearly any optimization problem. If a task takes 100 seconds and doubling the processor speed cuts it to 90 seconds, then there was an improvement. If doubling the speed or halving the latency of the storage device in the same system cuts the total execution time from 100 seconds to 70 seconds, you'd say that there was more benefit from the storage component upgrade than from the processor upgrade. Develop performance functions for both, multiply by prices to reduce the dimensions to a single independent variable (cost), combine to create one performance vs. cost equation, do a quick bit of math to find the minimum unit cost, and you've solved the optimization problem.

[Starfalcon]

Something cannot be both I/O bound and CPU bound. It has to be one or the other. Being bound by something means that you cannot improve performance without improving that thing.

How so? If I have a rig with both a slow CPU and slow hard drive running massive photoshop files, I think that would qualify as bound by both CPU and I/O. The CPU would be bound by being unable to process all the filters properly, and the I/O in that the hard drive would be unable to load the files to the ram and CPU to process.

[Shining Arcanine]

Something cannot be both I/O bound and CPU bound. It has to be one or the other. Being bound by something means that you cannot improve performance without improving that thing.

How so? If I have a rig with both a slow CPU and slow hard drive running massive photoshop files, I think that would qualify as bound by both CPU and I/O. The CPU would be bound by being unable to process all the filters properly, and the I/O in that the hard drive would be unable to load the files to the ram and CPU to process.

In your example, the program is I/O bound when loading and CPU-bound at processing. It is never both I/O-bound and CPU-bound at the same time. If it were, it would mean that everything is running perfectly, as you would not have one resource causing a stall in the utilization of another resource.

[JustAnEngineer]

In your example, the program is I/O bound when loading and CPU-bound at processing. It is never both I/O-bound and CPU-bound at the same time.

I agree. While performance for each tiny fraction of a second is limited by just one element, the overall program performance is a combined function of all of them. There is a percentage that could be improved with storage system upgrades and a percentage that could be improved with processor upgrades. Let's be willing to accept some imprecision and say that performance is "mostly" bound by one or the other or that it depends strongly on two of them.

[Zoomastigophora]

Er...is there any chance we could not let the SA train plow through this thread? I'd genuinely like donuts to test those scenarios I listed if he has the time. I do some encoding as well and if putting the source on the RAM disk speeds up the operation then I will probably start doing so in the future.

[ChronoReverse]

Actually, I'm pretty sure the reason he's seeing these anomalous results is this:

I am just using the default encoding settings to (RF of 20) etc. and encoding into a mkv file and using AC3 and DTS passthrough. I do have six cores @ 3.75 so with DVD's it fluctuates depending on the quality and length of the DVD if all cores are 100%. When I said they are not 100% I mean a core or two could dip into the 85-95% range briefly every once in a while

Basically since he's straight copying a significant portion of the video/audio, the process ended up I/O bound instead of CPU bound. This is further exacerbated by his lower encoding settings coupled with six core CPU (one can easily get faster than real-time encoding with the lower x264 settings - which still yields better quality than using CUDA but that's another story).

The smoking gun is that the CPU usage can varied from something other than pegged at 100%.

[Zoomastigophora]

The smoking gun is that the CPU usage can varied from something other than pegged at 100%.

That's some good grammar there

But yes, if his x264 settings are sufficiently low then the encoding process becomes I/O bound. Still, you'd have to be at pretty low settings for that to happen. Anyone know what handbrake defaults the x264 settings to?

[harleyca]

An old thread but I must add my 2 cents.....

I have 16GB of RAM, I use handbrake alot and I have a Phenom II X6 1100T CPU, that's 6 x 3.3GHz and I use 7200 rpm hard drives. I generally have my source and destination on the same hard disk and run handbrake on its regular/normal preset. My cpu runs at 98-100% virtually always whilst encoding except when I start doing stuff in the background that requires hard disk activity then the cpu usage will slow down. What I'm getting at is.... the cpu is generally the bottle neck not the hard disk unless your moving large files around whilst encoding therefore there is no benefit of using a RAM disk. It takes about a minute to copy 7GB of data from the same source drive to destination drive yet encoding that much data takes about 15 minutes, the cpu is definitely the bottle neck and there is little if any benefit of using a ram disk.