Bcache Testing: IOPS

Previously we looked at the throughput performance of bcache by running IOzone on a common SATA disk, an Intel X25-E SSD, and Bcache using the SSD to cache a single drive. This article explores the IOPS performance of the same configuration hoping to find areas where bcache might shine.

In a recent article I presented two new patch concepts (bcache and flashcache) for improving disk performance by using SSD’s as a caching mechanism for hard drives. In reality the caching is achieved by using a block device to cache another block device, but practically it’s using an SSD to cache hard drives. I’ve been waiting for some time for a patch that increases performance by using a cache that is larger than the disk cache and faster than the disk. SSD’s fit that bill pretty well, especially given their fantastic read performance compared to disks.

The second article in the series examined the throughput performance of the bcache patches applied to the 2.6.34 kernel. The performance was not what was expected but that was actually expected (clear as mud – right?). These are early patches and don’t have the performance at this time and have not been tuned for certain workloads. Consequently, performance was not expected to be really good but there were a few bright spots in the throughput performance exploration.

This article is the next in the series and examines the IOPS performance of the four storage configurations:

  1. Single SATA II disk (7,200 rpm 500GB with 16MB cache)
  2. Single Intel X25-E SLC disk (64GB)
  3. Bcache combination that uses the Intel X25-E as a cache for the SATA drive and uses the CFQ (Completely Fair Queuing) IO Scheduler that is the default for most distributions
  4. Bcache combination that is the same as the previous but uses the NOOP IO Scheduler for the SSD that many people think could help SSD performance.

The details of the configurations are below as are the details of the benchmarks and tests run. Once again, we will be using our good benchmarking techniques in this article.

For measuring IOPS performance I’m going to use IOzone. It has switches that turn measurements into operations per second (IOPS).

IOzone and IOPS

IOzone is one of the most popular throughput benchmarks. It’s open-source and is written in very plain ANSI C (not an insult but a compliment). It is capable of single thread, multi-threaded, and multi-client testing. The basic concept of IOzone is to break up a file of a given size into records. Records are written or read in some fashion until the file size is reached. While it is more commonly used for measuring throughput performance, it can also measure operations per second (IOPS – IO Operations Per Second). More specifically, it can be used to measure sequential read and write IOPS as well as random read and random write IOPS.

For this article, IOzone runs four specific tests and computes IOPS for each. The four tests are:

  • Write
    This is a fairly simple test that simulates writing to a new file. Because of the need to create new metadata for the file, many times the writing of a new file can be slower than rewriting to an existing file. The file is written using records of a specific length (either specified by the user or chosen automatically by IOzone) until the total file length has been reached.
  • Read
    This test reads an existing file. It reads the entire file, one record at a time.
  • Random Read
    This test reads a file with the accesses being made to random locations within the file. The reads are done in record units until the total reads are the size of the file. The performance of this test is impacted by many factors including the OS cache(s), the number of disks and their configuration, disk seek latency, and disk cache among others.
  • Random Write
    The random write test measures the performance when writing a file with the accesses being made to random locations with the file. The file is opened to the total file size and then the data is written in record sizes to random locations within the file.

For IOzone the system specifications are fairly important. In particular, the amount of system memory is important because this can have a large impact on the caching effects. If the problem sizes are small enough to fit into the system or file system cache (or at least partially), it can skew results, even for IOPS testing. Comparing the results of one system where the cache effects are fairly large to a system where cache effects are not large, is comparing the proverbial apples to oranges. For example, if you run the same problem size on a system with 1GB of memory versus a system with 8GB you will get much different results.

As with the throughput tests, the IOPS tests used a file size that is twice the size of memory. The goal is to push the file size out of what could be cached by Linux. However, the actual record size within the file is under control of the user so we can effectively prescribe the size of each IO operation when performing the IOPS testing.

For this article a total file size of 16GB was used. Within this 16GB file size, five record sizes are tested: (1) 8KB, (2) 32KB, (3) 64KB, (4) 128KB, and (5) 512KB record sizes. These sizes were chosen because the run times for smaller record sizes were much longer and using our good benchmarking skills of running each test 10 times, resulted in very long benchmark times (weeks). You might laugh at the larger record sizes, but there are likely to be applications that depend upon how quickly they can read/write 512KB records (I quit saying “never” with respect to application IO – I’ve seen some truly bizarre patterns so “never” has been removed from vocabulary.).

The command line for the first record size (8KB) is,

./iozone -Rb spreadsheet_output_8K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_8K.txt

The command line for the second record size (32KB) is,

./iozone -Rb spreadsheet_output_32K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_32K.txt

The command line for the third record size (64KB) is,

./iozone -Rb spreadsheet_output_64K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_64K.txt

The command line for the fourth record size (128KB) is,

./iozone -Rb spreadsheet_output_128K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_128K.txt

The command line for the fifth record size (512KB) is,

./iozone -Rb spreadsheet_output_512K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_512K.txt

Test System

The tests were run on the same system as previous tests. The system highlights of the system are:

  • GigaByte MAA78GM-US2H motherboard
  • An AMD Phenom II X4 920 CPU
  • 8GB of memory (DDR2-800)
  • Linux 2.6.34 kernel (with bcache patches only)
  • The OS and boot drive are on an IBM DTLA-307020 (20GB drive at Ultra ATA/100)
  • /home is on a Seagate ST1360827AS
  • There are two drives for testing. They are Seagate ST3500641AS-RK with 16 MB cache each. These are /dev/sdb and /dev/sdc.

Only the second Seagate drive was used, /dev/sdc, for the file system. Since the version of bcache I used could not yet cache a block partition, I used the whole device for the file system (/dev/sdc).

Comments on "Bcache Testing: IOPS"

I loved your blog post. Awesome.

Outstanding story there. What happened after?
Take care!

Fantastic blog you have here but I was curious if you knew of
any forums that cover the same topics talked
about in this article? I’d really like to be a part of community where I can get
feed-back from other experienced people that share
the same interest. If you have any recommendations,
please let me know. Many thanks!

Here is my web page: forskolin extract (http://www.mep.aw)

I am so grateful for your article.Thanks Again. Cool.

Thank you for every other informative web site. Where else could I am getting that kind of info written in such an ideal means?
I’ve a undertaking that I’m just now running on,
and I’ve been on the glance out for such info.

Review my web blog: credit monitoring services (http://www.instructables.com)

When someone writes an article he/she keeps the image of a user in his/her mind that how a user can be
aware of it. Thus that’s why this piece of writing is great.

What’s up, its fastidious post about media print,
we all understand media is a wonderful source of information.

Visit my page – acne skin treatment; Carmon,

Howdy! I know this is kinda off topic but I’d figured I’d ask.
Would you be interested in exchanging links or maybe guest writing a blog article or
vice-versa? My blog discusses a lot of the same topics as yours and I feel
we could greatly benefit from each other. If you’re interested feel free to shoot me an e-mail.
I look forward to hearing from you! Awesome blog by the way!

my homepage – credit fraud victim

I’m not that much of a online reader to be honest but your blogs really nice,
keep it up! I’ll go ahead and bookmark your website to come back down the
road. Many thanks

Here is my homepage dispute credit report findings

If some one needs expert view about blogging and site-building afterward i propose him/her to pay a quick visit this web site, Keep up the good job.

Also visit my web page http://www.buzzfeed.com

Very handful of internet sites that happen to be detailed below, from our point of view are undoubtedly well really worth checking out.

We came across a cool web page which you may take pleasure in. Take a appear if you want.

Very good blog you have here but I was wanting to know if you
knew of any message boards that cover the same topics
discussed in this article? I’d really like to be a part of group where I can get responses
from other experienced individuals that share the same
interest. If you have any suggestions, please let
me know. Cheers!

Feel free to visit my webpage: forskolin diabetes

Please let me know if you’re looking for a author for
your site. You have some really good posts and I think I would be a good
asset. If you ever want to take some of the load
off, I’d absolutely love to write some material for your blog in exchange for a link
back to mine. Please shoot me an email if interested.
Many thanks!

My web-site :: where is forskolin sold [lightofzion.org]

Hi there to every one, the contents present at this web page are really remarkable for people knowledge,
well, keep up the good work fellows.

Stop by my web-site: thirdsector.mlog.taik.fi

Peculiar article, exactly what I wanted to find.

Have a look at my web-site; where to buy forskolin 125mg (fortnassau.com)

Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something.
I think that you could do with some pics to drive the message home a little bit, but other than that, this is excellent blog.
A fantastic read. I’ll definitely be back.

my weblog forskolin for weight loss

Hello would you mind sharing which blog platform you’re using?
I’m planning to start my own blog soon but I’m having a tough time making a decision between BlogEngine/Wordpress/B2evolution and Drupal.
The reason I ask is because your design and style seems different then most blogs and I’m looking for something unique.

P.S Sorry for getting off-topic but I had to ask!

Here is my weblog; forskolin extract, https://www.uab.edu/give/campaign/component/k2/item/139-make-plans-to-attend-crowdfunding-101?start=20,

Wonderful article! That is the type of info that should be shared around the net.

Shame on the seek engines for no longer positioning this put up higher!
Come on over and discuss with my web site . Thank you =)

Also visit my web page :: acidophilus dietary supplement

Thank you ever so for you blog article.Really looking forward to read more. Will read on…

It’s hard to find educated people for this subject, but you
sound like you know what you’re talking about! Thanks

I do trust all the concepts you’ve presented to your post.
They’re very convincing and can certainly work. Still,
the posts are too short for novices. May you please extend them a little from next time?
Thank you for the post.

Feel free to visit my web-site – LATaxNetwork Inc.

Hi there, You’ve done a great job. I’ll definitely digg it and personally suggest to my friends.
I’m sure they will be benefited from this site.

Also visit my webpage; credit fraud analyst salary (Guadalupe)

If some one desires expert view concerning blogging and site-building afterward i propose him/her to visit this weblog, Keep up the pleasant job.

my blog; enhanced identity theft protection website

Your style is very unique in comparison to other people I’ve read stuff from.
Thank you for posting when you’ve got the opportunity,
Guess I will just bookmark this web site.

Leave a Reply