Tech Brief: RAID for Number Crunchers

Posted on January 16, 2011 by

3


SuperMicro Mini-Tower With 4 Hot-Swappable Drives

Disk access is a big bottleneck for the kinds of number crunching we do at Lucena Research. And in order to optimize our hardware setup I’ve been spending a lot of time researching RAID.  Here’s what you need to know. (see a more recent post about RAID with performance results for Ubuntu Linux).

read related articles

Why RAID?

RAID stands for Redundant Array of Independent Disks. RAID is a technology that enables a computer to combine multiple hard disks into a single storage volume.  Your software sees that volume as a single logical disk.   When compared to a single physical disk, RAID can offer significantly increased speed and reliability for data storage.

RAID can offer significantly increased speed and reliability for data storage

The advantages RAID offers can be significant.  As an example, in our shop, we read large chunks of data into RAM, then crunch on it for minutes at a time.  For one particular simulation we currently spend about 180 seconds to read data in that we process for about 60 seconds.  We repeat this operation many times as we test various scenarios. With the proper RAID configuration we may be able to cut the read time to 45 seconds (one 1/4 of the original time).  That will enable us to run many more scenarios.

Another important benefit is reliability.  Some RAID configurations implement what amounts to a live, always on backup that can sustain one or more physical disk failures. In some cases the defective drives can be replaced without shutting down the computer (called a hot swap).

Hardware & Cost

In the past it was necessary to purchase a separate controller card if you wanted RAID. But now most Intel-based desktop motherboards ship with a RAID chip on board.  As of this writing the P55 Express Chipset is one of the more popular implementations. I’m going to describe what you can do with a P55 based motherboard, but similar capabilities are available with other chipsets and other manufacturers. (update: watch out for fake RAID)

Besides the added cost of controller card, or motherboard with RAID support, you must purchase additional disks.  For instance, to support a 4 disk RAID volume, you must purchase 4 x 500GB physical disks.  Fortunately, another advantage of RAID is that smaller disks are usually less likely to fail.

As a rule of thumb, you can expect a RAID volume to cost about 2x as much as a single drive of the same size because you’ll need 4x as many drives that cost about 1/2 as much. At this writing, for instance, an enterprise quality 500GB drive runs about $75 (total $300), while a single 1000GB drive runs $150.

Which RAID Configuration is Best?

That depends on your application of course.  Is read speed more important that write speed?  Is reliability more important than I/O performance? Depending on your priorities, you can select from a number of RAID “levels” that describe the way data is sliced and diced among the disks to provide the various types of RAID benefits you want. As of this writing, available RAID levels include: 0, 1, 2, 3, 4, 5, 6, 1+0, and 0+1 (I’ll skip the gory details for each level, but you can learn more here).

The important thing to know is that I/O speed is gained by splitting data in to “slices” and then writing the slices in parallel to several disks at once — this is called striping. And there are two different methods for adding reliability: 1) Mirroring, where the data is duplicated on several disks, and 2) Parity, where additional bits are written to the disks, so missing data can be recovered if a disk is lost.  Whether you use mirroring or parity the data can be recovered if a disk fails; some configurations can sustain more than one disk failure. The various RAID levels implement striping and redundancy in different ways.

Motherboards with the P55 chip support RAID levels 0, 1, 5, and 1+0, and these are also the most popular.  Let’s compare theoretical performance assuming 4 x 250GB disks in a RAID array:

RAID Level Available Data Read Speedup Write Speedup Fault Tolerance
0 2,000GB 4x 4x 0 disks
1 500GB 4x 1x 3 disks
5 1,500GB 3x depends 1 disk
1+0 1,000GB 4x 2x 2 disks

RAID 5 write speed up depends on the time cost of the parity calculation that must be completed during writes.  In practice, the impact can be minimal, providing a 3x speedup.

The Winner: RAID 1+0

For my application, I think RAID 1+0 will be the way to go because we spend a lot of time reading data and it offers 4x speedup there.  It also offers significant fault tolerance, and moderate write speedup as well.

Well, Almost…

The chart above reports theoretical performance.  The real performance we see is a consequence of a combination of hardware and software interactions, including disk cache size, and memory bandwidth.  The only way to know for sure for a particular setup is to experiment.  So, I’ll followup with results from our own servers. (see the post with real numbers)

Disclosure:  All comments are provided for general interest only and should not be construed as investment advice.

Advertisements