Personal computing discussed

Moderators: renee, morphine, Steel

 
emorgoch
Gerbil Elite
Topic Author
Posts: 719
Joined: Tue Mar 27, 2007 11:26 am
Location: Toronto, ON

Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 12:37 pm

I've looked through the last few SSD articles, and haven't been able to come up with the answer to this questions:
What's the IOMeter profiles used for file server, web server, workstation, and database tests?

Mostly interested because I need to set up an IOMeter run to benchmark a couple of the systems here in the office, and I'm curious how it's profile sits compared to others.
Intel i7 4790k @ stock, Asus Z97-PRO(Wi-Fi ac), 2x8GB Crucial DDR3 1600MHz, EVGA GTX 1080Ti FTW3
Samsung 950 Pro 512GB + 2TB Western Digital Black
Dell 2408WFP and Dell 2407WFP-HC for dual-24" goodness
Windows 10 64-bit
 
Ryhadar
Gerbil XP
Posts: 463
Joined: Tue Oct 21, 2008 9:51 pm

Re: Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 12:45 pm

If I recall correctly, those specifications are in the the documentation for iometer found here: http://www.iometer.org/
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 3:58 pm

Some of the TR ones were deemed not as relevant anymore and have been removed for a while. I had another thread asking the same thing and no one replied. At one point I actually remade those patterns based on the older descriptions from StorageReview.com, plus adding the patterns that they have added to complete a suite of my own. I forgot where I put them though.

Which patterns are you specifically interested in?
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
emorgoch
Gerbil Elite
Topic Author
Posts: 719
Joined: Tue Mar 27, 2007 11:26 am
Location: Toronto, ON

Re: Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 4:07 pm

Specifically the database one. No big deal. Just curious since TR is pretty good about their testing methodologies, I was finding it curious that I couldn't find the actual profiles they were using.
Intel i7 4790k @ stock, Asus Z97-PRO(Wi-Fi ac), 2x8GB Crucial DDR3 1600MHz, EVGA GTX 1080Ti FTW3
Samsung 950 Pro 512GB + 2TB Western Digital Black
Dell 2408WFP and Dell 2407WFP-HC for dual-24" goodness
Windows 10 64-bit
 
Dissonance
Gerbil Elite
Posts: 535
Joined: Wed Dec 26, 2001 7:00 pm
Location: Vancouver, BC
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 4:46 pm

IIRC, the file and web results were part of the original IOMeter package, while the database and workstation patterns came from StorageReview. I was going to summarize them here but, looking at them now, I see an awful lot of variables to detail. Shoot me an email and I can send you a copy of the config file we use. Open that in IOMeter, and you can edit the individual access patterns to see how the I/Os are distributed.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 06, 2012 5:42 pm

I believe I pretty much repro'ed the test patterns plus the newer SR.com ones, but I lost the rapidshare links. :(

I think the database pattern is a lot of random stuff, may be like 70% random I/O and the rest sequential, or something.
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Thu Feb 09, 2012 10:44 am

OK, I managed to dig up stuff that I was planning to release to TR forums dated 2009/11! (procrastination at its best) :o

The database pattern that you are looking for is, as SR.com stated when they updated to Testbench3, pretty arbitrary. I personally think it is pretty lame. It is 100% access (1 row) with 8KB size where 67% is reads and 100% random. Instead of trying attach the files to web file lockers and stuff, I have decided to post my work here. It was back then when I got my shiny new 785G+SB710 and I wanted to compare it to ICH9 and stock MS vs Intel/AMD drivers.

So here's the readme that I wrote up for the zip file package (I have updated the links for reference):
TR Forums Iometer Workload Patterns
-----------------------------------
(First created: 2009/11/24)

Files:
1. trforums_single.icf - all patterns with 1 worker (single CPU core)
2. trforums_multi.icf - all patterns with multiple workers (1 per CPU core)

Access specifications:
1. Idle (original)
2. Default (original)
3. All in one (original)
4. File Server - Intel/StorageReview.com
   http://www.storagereview.com/articles/200003/20000313OSandBM_5.html
5. Web Server - StorageReview.com
   http://www.storagereview.com/articles/200111/20011109Renaissance_4.html?page=0,7
6. Database - Read - Intel/StorageReview.com
   http://www.storagereview.com/articles/200003/20000313OSandBM_5.html
7. Database - Write - Mirror image of "Database - Read"
8. Workstation - SR - StorageReview.com
   http://ixbtlabs.com/articles/hddide2k1feb/iometer.html
9. Workstation - X-bit labs (GReY)
   http://www.xbitlabs.com/articles/storage/display/sil-3124_3.html#sect0
10. Streaming Writes - THG
    http://www.tomshardware.com/reviews/x25-e-ssd-performance,2365-7.html
11. Streaming Reads - THG
    http://www.tomshardware.com/reviews/x25-e-ssd-performance,2365-7.html

Test setups:
1. Run time: 10 minutes
2. Ramp up time: 20 seconds
3. Outstanding I/O's: 1, 2, 4, 8, 16, 32, 64, 128, 256 (9 tests)
4. Number of workers: controlled by file
The "test setups" were my own definition for my own testing. If we use the same setups then we can compare our results, or at least that was what I was hoping. Configs #4 and up are my own recreation based on the links. I have re-created the older (and deprecated) patterns, plus I added new patterns (#9-11) that I thought to be useful at the time. Why 2 .icf files? More on that later.

trforums_single.icf file. Just copy and paste in your favourite text editor.
Version 2006.07.27 
'TEST SETUP ====================================================================
'Test Description
   TR Forums - Single Core CPU
'Run Time
'   hours      minutes    seconds
   0          10         0
'Ramp Up Time (s)
   20
'Default Disk Workers to Spawn
   1
'Default Network Workers to Spawn
   NUMBER_OF_CPUS
'Record Results
   ALL
'Worker Cycling
'   start      step       step type
   1          1          LINEAR
'Disk Cycling
'   start      step       step type
   1          1          LINEAR
'Queue Depth Cycling
'   start      end        step       step type
   1          256        2          EXPONENTIAL
'Test Type
   CYCLE_OUTSTANDING_IOS
'END test setup
'ACCESS SPECIFICATIONS =========================================================
'Access specification name,default assignment
   Default,NONE
'size,% of size,% reads,% random,delay,burst,align,reply
   2048,100,67,100,0,1,0,0
'Access specification name,default assignment
   All in one,NONE
'size,% of size,% reads,% random,delay,burst,align,reply
   512,5,100,0,0,1,0,0
   512,5,75,0,0,1,0,0
   512,5,50,0,0,1,0,0
   512,5,25,0,0,1,0,0
   512,5,0,0,0,1,0,0
   4096,5,100,0,0,1,0,0
   4096,5,75,0,0,1,0,0
   4096,5,50,0,0,1,0,0
   4096,5,25,0,0,1,0,0
   4096,5,0,0,0,1,0,0
   16384,5,100,0,0,1,0,0
   16384,5,75,0,0,1,0,0
   16384,5,50,0,0,1,0,0
   16384,5,25,0,0,1,0,0
   16384,5,0,0,0,1,0,0
   32768,5,100,0,0,1,0,0
   32768,5,75,0,0,1,0,0
   32768,5,50,0,0,1,0,0
   32768,5,25,0,0,1,0,0
   32768,5,0,0,0,1,0,0
'Access specification name,default assignment
   File Server,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   512,10,80,100,0,1,0,0
   1024,5,80,100,0,1,0,0
   2048,5,80,100,0,1,0,0
   4096,60,80,100,0,1,0,0
   8192,2,80,100,0,1,0,0
   16384,4,80,100,0,1,0,0
   32768,4,80,100,0,1,0,0
   65536,10,80,100,0,1,0,0
'Access specification name,default assignment
   Web Server,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   512,22,100,100,0,1,0,0
   1024,15,100,100,0,1,0,0
   2048,8,100,100,0,1,0,0
   4096,23,100,100,0,1,0,0
   8192,15,100,100,0,1,0,0
   16384,2,100,100,0,1,0,0
   32768,6,100,100,0,1,0,0
   65536,7,100,100,0,1,0,0
   131072,1,100,100,0,1,0,0
   524288,1,100,100,0,1,0,0
'Access specification name,default assignment
   Database - Read,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   8192,100,67,100,0,1,0,0
'Access specification name,default assignment
   Database - Write,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   8192,100,33,100,0,1,0,0
'Access specification name,default assignment
   Workstation - SR,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   8192,100,80,80,0,1,0,0
'Access specification name,default assignment
   Workstation - X-bit labs,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   512,1,0,100,0,1,0,0
   1024,2,0,100,0,1,0,0
   2048,1,0,100,0,1,0,0
   4096,50,60,80,0,1,0,0
   8192,4,50,100,0,1,0,0
   16384,6,50,100,0,1,0,0
   20480,2,50,100,0,1,0,0
   24576,2,50,100,0,1,0,0
   28672,1,50,100,0,1,0,0
   32768,13,70,70,0,1,0,0
   49152,1,50,100,0,1,0,0
   53248,1,50,100,0,1,0,0
   65536,14,80,60,0,1,0,0
   66048,2,50,100,0,1,0,0
'Access specification name,default assignment
   Streaming Writes - THG,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   65536,34,100,0,0,1,0,0
   131072,33,100,0,0,1,0,0
   262144,33,100,0,0,1,0,0
'Access specification name,default assignment
   Streaming Reads - THG,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   65536,34,0,0,0,1,0,0
   131072,33,0,0,0,1,0,0
   262144,33,0,0,0,1,0,0
'END access specifications
Version 2006.07.27
At the time, I did not know if single vs multiple core CPUs would make a difference in the benchmarks, so I created 2 profiles. To spawn as many workers as CPUs, just change the "Default Disk Workers to Spawn" field to "NUMBER_OF_CPUS" (minus quotes). I think what I found was that it made very little, if at all any, difference.

Hope this helps.

PS. I actually made a summary with all my results back then, ready for forum posting. I did a lot of combos (ICH9 vs SB710, stock IDE vs stock AHCI vs vendor AHCI drivers, Win7 vs Win2K3)! :o :o
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Fri Feb 01, 2013 12:20 pm

Thank you. Been lookin' around for something like this.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Fri Feb 01, 2013 12:27 pm

So number of workers... what do people tend to use? I've seen someone say the more the better, 10 as a minimum. You indicate 1 per CPU or CPU core. I got 10 working right now on a quad core CPU and it's barely 10% CPU usage.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Fri Feb 01, 2013 12:38 pm

Ok ok ok ok wait I think I got this. So when TR makes a graph like these: http://techreport.com/review/23990/sams ... reviewed/8

...it runs a test with 1 worker for a while, records the numbers, then runs with 2, then 4, then 8, et cetera? I've seen someone say with SSDs you have to wait a while, like 8 hours or overnight, to get a valid measure. Is there any conventional wisdom about how long you should wait with this stuff and if it's different with SSD versus SATA/SAS?
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Fri Feb 01, 2013 12:40 pm

Scrotos wrote:
So number of workers... what do people tend to use? I've seen someone say the more the better, 10 as a minimum. You indicate 1 per CPU or CPU core. I got 10 working right now on a quad core CPU and it's barely 10% CPU usage.

I don't know anymore. At the time I thought each worker will be one thread, so I assign 1 worker per CPU core. I think the results turn out to be the same.

Not sure if #workers = #concurrent requests.
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
Dissonance
Gerbil Elite
Posts: 535
Joined: Wed Dec 26, 2001 7:00 pm
Location: Vancouver, BC
Contact:

Re: Tech Report's IOMeter Test Paterns

Fri Feb 01, 2013 3:44 pm

Our IOMeter tests run with one worker. They start with one concurrent I/O request and then ramp up from there. The test runs for three minutes for each I/O request tier. The number of workers is separate from the number of concurrent requests.

Before any tests are run, IOMeter creates a single file that spans the entire capacity of the drive, effectively putting it into a used state. We secure erase drives before running IOMeter in the first place, so they all have a fresh-state starting point. There's no need to let the drives sit in this case, since they're freshly wiped and there's no data for the background garbage collection routines to move around.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 10:01 am

I wish you'd just post your test file(s) or, if you have described the methodology previously with that info, link to it in your SSD "how we test" section. It'd make it easier for schmoes like me who are interested in doing similar tests but aren't too familiar with the tools. For instance, I'm trying to get some decent numbers on this: viewtopic.php?f=29&t=86144

I less want to become an expert in benchmarking and more would like a relatively easy way to just get numbers and figure out if RAID5 is going to perform better than RAID6 in my scenario, i.e. if the controller is saturated anyway and it don't much matter. Either way, I don't see a reason to not post the config file y'all use for reference in the articles. After all, "Most of the tests and methods we employed are publicly available and reproducible." ;)
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 10:15 am

The patterns TR used were not customized. Most of them were public info at one point or the other. Some patterns were removed in later distributions of IOMeter so what I did was just searching around and based on SR and Toms (!) older articles I found+reconstructed most if not all of them. I only added a couple in there from the established one. SR actually mentioned that the "workstation" pattern (or was it the database one?) is actually quite useless.

But yes, posting the file would be convenient. Although TR is not in the business of teaching you how to run IOMeter.
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 12:03 pm

Yes, I'm aware of that. Thanks for the reprimand that I thought I'd already addressed. I do find it odd, though, since you seem to have spent quite some time trying to recreate TR's testing methodology yourself. I'd have thought you'd be more like, "yeah, post that biznatch!" rather than "RTFM."

I do wholeheartedly thank you for the work you did in trying to recreate the test patterns that TR uses and especially for digging up work from 3 years prior and making it publicly available! The currently available version of IOMeter has entries like:

16K; 50% Read; 0% random
16K; 25% Read; 0% random
16K; 0% Read; 0% random

...which don't seem to map to the entries you had like:

Database - Read
Database - Write

I wouldn't even know where to start looking to try and dig up various profiles that are publically available going back 6 years or longer; even with the links you provided it seemed like a ton of reading for something I was hoping would be a quick "run this profile and find your IOps" deal. Heck, you even spent a decent amount of time researching all this and you don't even know how TR gets their concurrent requests numbers.

So again, thank you for having taken the time to get some useful profiles and information out there.
 
Dissonance
Gerbil Elite
Posts: 535
Joined: Wed Dec 26, 2001 7:00 pm
Location: Vancouver, BC
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 12:30 pm

Feel free to shoot me an email if you'd like a copy of our IOMeter test file. The thing is, that file just defines the access patterns (request size, read/write distribution, randomness, etc...). You still have to configure the workers, concurrent I/O scaling, and test length manually.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 12:41 pm

Scrotos wrote:
Thanks for the reprimand that I thought I'd already addressed. I do find it odd, though, since you seem to have spent quite some time trying to recreate TR's testing methodology yourself. I'd have thought you'd be more like, "yeah, post that biznatch!" rather than "RTFM."
Sorry if I came across that way. It was not my intention. I actually think the patterns are the easy part, since they are documented on the web. (see below)

Scrotos wrote:
I do wholeheartedly thank you for the work you did in trying to recreate the test patterns that TR uses and especially for digging up work from 3 years prior and making it publicly available! The currently available version of IOMeter has entries like:

16K; 50% Read; 0% random
16K; 25% Read; 0% random
16K; 0% Read; 0% random

...which don't seem to map to the entries you had like:

Database - Read
Database - Write

I wouldn't even know where to start looking to try and dig up various profiles that are publically available going back 6 years or longer
Basically it seemed like once Intel decided to make IOMeter open source, the distributions stopped including what Intel included before in terms of the profiles. It was unfortunate. Good thing the patterns were relatively easy to recreate. And subsequent analysis by SR debunked some patterns as not that useful anyway.

Scrotos wrote:
even with the links you provided it seemed like a ton of reading for something I was hoping would be a quick "run this profile and find your IOps" deal. Heck, you even spent a decent amount of time researching all this and you don't even know how TR gets their concurrent requests numbers.
Dissonance wrote:
Feel free to shoot me an email if you'd like a copy of our IOMeter test file. The thing is, that file just defines the access patterns (request size, read/write distribution, randomness, etc...). You still have to configure the workers, concurrent I/O scaling, and test length manually.
All that worker, concurrent I/O, test length stuff, IMO, are the "biznatch" in terms of the setup. :P The clunky UI does not help things either. There was no documentation and so I just picked my own settings. I can never really compare with TR's numbers (not to mention I have different CPU+mobo). So I only made comparisons based on my own baselines. That was when I realize that I don't really need TR's exact numbers, just the trends that I needed to match.

But yes, if any of us can post a quick "how to use IOMeter" guide, I am all for it. The strength in TR is not just the articles, but the forum community as well. Let's crowdsource this! :D
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Mon Feb 04, 2013 4:31 pm

Oi, email sent.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Tue Feb 05, 2013 2:45 pm

Ok, got it, thanks. So I see under Test Setup where to change the time. The # of Outstanding I/Os is the concurrent requests, I assume.

Looking at the CSVs that get generated, I see Queue Depth (is Concurrent I/O Requests is Outstanding I/Os?) and IOps which seems to be the Read IOps and Write IOps combined.

You plot IOps versus Queue Depth? I ask this because I ran the database test using 10 minute length up to 256 depth and got these IOps numbers:

1 5409
2 6650
4 6593
8 6650
16 6618
32 6589
64 6620
128 6596
256 6431

Ok, so the sucker plateaued after 2 concurrent requests on my workstation (Q6600, 4 GB RAM, XP SP3, SSD is my main hard drive) and I'm fine with that. I'm just trying to get an idea of how the tool works. But what concerns me is that my starting point is a few thousand higher than in the TR test. Is it because I didn't secure erase the entire drive? The sucker only has less than 40 GB free so I figure it's been mostly written to, ya? Or is something else going on here?

I know the test results aren't directly comparable but I would fully expect my results to be lower, not higher. I'm on some old G31/G33 class motherboard. I think it's SATA 2 at best? And using the computer for other stuff while benchmarking.

Maybe Cycling Options? What should that be? It is defaulting to Cycle # Outstanding I/Os. I also noticed I had # of workers to spawn automatically set to # of CPUs. I'll set that to 0 and see what happens.

BTW, am running it on a server now to see if it ramps further and it's been sitting there for quite some time. Oh lordy, it's got a 680 GB file and counting for this test run! I'll pick that up tomorrow.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Wed Feb 06, 2013 2:13 pm

Hrm, so 650 GB test file on one system, 1.1 TB test file on the other. 8 KB free space. Heh. Something didn't work right, though.

SSD array:

1 274
2 287
[nothing else posted results, just a "results" line with no content]

SAS array:

1 226
2 356
4 497
8 641
16 779
[blank "results" for the next several]

I guess the system ran out of space to write the results file? I didn't have this issue with my workstation I tried this on.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Wed Feb 06, 2013 4:01 pm

http://old.nabble.com/How-many-workers- ... 04637.html

Good info there. So the first test runs I did made a 10 GB iobw.tst on the SSD and SATA drives on my workstation. Why, I don't know, since I didn't limit the size at all. I'm going to try using this sucker to make some 100 GB files: http://blog.open-e.com/a-few-practical- ... t-iometer/

I'm assuming TR runs on unpartitioned raw drives for their testing so iobw.tst is not a factor in their testing, but since I'm doing it on actual filesystems it's something I have to care about. Some of the blah blah blah I read indicates that the file size should be more than twice the system's RAM so that no filesystem caching will affect the testing. So I'll rerun the server tests and see what I get. Haven't yet delved into the whole number of workers thing or messing with the RAID controller's cache.

Duh.

Dissonance wrote:
Before any tests are run, IOMeter creates a single file that spans the entire capacity of the drive, effectively putting it into a used state.


So that's a bit of a bind for me since I'm running on OS drives. How much space do I leave empty, then? 1 GB? Will the OS decide to dump some temp stuff to disk during testing? Ugh. Will revisit.
Last edited by Scrotos on Wed Feb 06, 2013 6:15 pm, edited 1 time in total.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Wed Feb 06, 2013 5:11 pm

So that program I posted that made iobw.tst files seems to not work with the current version of iometer. I ran a test with 10,000,000 sectors which worked out to be 5 GB. Rerunning with 200,000,000 sectors which should get me the 100 GB I wanted. BTW, when I re-ran the test with a larger number of sectors, it didn't change the size of the iobw.tst file. I had to manually delete it to get a new size created.

SAS, 5 GB test file, TR db test pattern:

1 387
2 637
4 953
8 1306
16 1680
32 1868
64 1883
128 1870
256 1873

SAS, 100 GB test file, TR db test pattern (letting it spawn workers to equal CPU cores gave no difference in performance):

1 313
2 494
4 701
8 917
16 1139
32 1338
64 1418
128 1411
256 1422

SAS, 100 GB test file, 8 workers, TR db test pattern:

1 916
2 1139
4 1341
8 1417
16 1422
32 1423
64 1422
128 1477
256 1521

SAS, 200 GB test file, TR db test pattern:

1 283
2 447
4 623
8 814
16 1010
32 1211
64 1331
128 1337
256 1331

So hey, makin' progress. I'm also using outstanding I/Os to be 256. (doesn't matter with this test) Wow, the iops take a dive with a larger test file. I think I have a handle on the test file size so will be working on the workers next. Ok workers ramps up to the limit quicker, but that's about it it seems.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Fri Feb 08, 2013 10:37 am

Flying Fox wrote:
But yes, if any of us can post a quick "how to use IOMeter" guide, I am all for it. The strength in TR is not just the articles, but the forum community as well. Let's crowdsource this! :D


I think I have a version of the TR benchmark that has all the settings pre-filled for what tests they run. I've shot off a copy to the bloke in the know to see if it's missing anything. If so, the how-to would be pretty easy! :D

I did find this: http://www.techpowerup.com/forums/showt ... p?t=166005

That fellow is testing other stuff than what TR tests. I am happy with the test patterns for databases that you and TR have. I have a wishlist item for IOMeter but it doesn't look like anyone's maintaining the code anymore.

Once I get the settings fixed up for the TR test in a way it's easy to reproduce, I've got questions on the 4k alignment. Wheeee.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Fri Feb 08, 2013 11:48 am

Actually, I'll pose my question in regards to 4K alignment now. Here are two different test patterns, one from TR, one from that TPU thread:

'Access specification name,default assignment
   File Server,DISK
'size,% of size,% reads,% random,delay,burst,align,reply
   512,10,80,100,0,1,4096,0
   1024,5,80,100,0,1,0,0
   2048,5,80,100,0,1,0,0
   4096,60,80,100,0,1,0,0
   8192,2,80,100,0,1,0,0
   16384,4,80,100,0,1,0,0
   32768,4,80,100,0,1,0,0
   65536,10,80,100,0,1,0,0

'Access specification name,default assignment
   File Server,ALL
'size,% of size,% reads,% random,delay,burst,align,reply
   512,10,80,100,0,1,4096,0
   1024,5,80,100,0,1,4096,0
   2048,5,80,100,0,1,4096,0
   4096,60,80,100,0,1,4096,0
   8192,2,80,100,0,1,4096,0
   16384,4,80,100,0,1,4096,0
   32768,4,80,100,0,1,4096,0
   65536,10,80,100,0,1,4096,0


Now the TR one has a 4K alignment set for the 512 byte pattern but nothing else in that profile. The TPU fellow has 4K alignment set for everything. It seems to me that you'd want to set 4K everywhere, wouldn't you? Why only the 512 byte size?

And where does 4K come into play? Vista and Win7 automatically make partitions aligned to 4K boundaries. SSDs like partitions aligned to 4K boundaries. Large drives using the "advanced format" 4K sectors... also like partitions aligned to 4K boundaries? So in both cases XP is a loser unless you make a partition using something 4K aware and then install XP later. So you'd want to align everything to 4K so the results would most closely match Vista/Win7 and OS X?

Someone in the TPU thread got errors on their tests when they included smaller than 4K data sizes with their 4K sectored drives. Does this happen with the TR test, too, but the error doesn't really show up? Or are the drives not 4K sectored so it doesn't matter? Or was that TPU guy having some other issue? I don't have any 4K sectored drives I can test this on to verify.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Wed Feb 20, 2013 1:55 pm

While waiting for dissonance to come back from vacation, I ran across this: http://www.tomshardware.com/reviews/ssd ... 226-5.html

Interesting stuff. Though looks like TR's already on top of that:

We run all our tests at least three times and report the median of the results. We've found IOMeter performance can fall off with SSDs after the first couple of runs, so we use five runs for solid-state drives and throw out the first two.


One thing that's odd is TH uses "LBA=full span" for iometer and I don't see that setting anywhere. I'm guessing they leave the sector size at 0 so it creates a test file that spans the entire drive? That's the only thing that makes sense to me.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Mon Feb 25, 2013 5:01 pm

So I think I have an all-in-one iometer config file that replicates TR's testing exactly. Waiting to hear back if it's ok to post. I don't see why it wouldn't be, but hey it's polite, innit?

I'll make another separate thread with a how-to for testing methodology with that. I'll also throw in some musings for those who want to test systems already in production like servers based on my experiences.
 
Scrotos
Graphmaster Gerbil
Posts: 1109
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Tech Report's IOMeter Test Paterns

Tue Feb 26, 2013 1:14 pm

Ok, posted my how-to here: viewtopic.php?f=5&t=86757

Thanks for all your help, hosers!

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On