Geeking Out on SSD Hardware Developments

In a prior article we dug deep into the anatomy of SSDs. That article started with the basics of how data “bits” are stored using floating gates, described how SSD chips are constructed, and covered the merits and problems with SSDs. SSDs hold great promise in terms of performance and power. However, as the article mentioned, there are some potential problems with SSDs that only now are being addressed by manufacturers.

SSD technology is evolving very quickly. This article will look at three SSD hardware developments: (1) Intel and Micron 25nm flash chips, (2) very large SSDs along with new PCIe SSD devices from someone other than Fusion-IO, and (3) a new OS independent device that can use an SSD as a read cache for a hard drive.

Intel and Smaller Cells

According to this link the theoretical lower limit on NAND cells appears to be around 20nm. One of the big reasons for this limitation is that as cells shrink in size they are, naturally, closer together. However, the voltage required to program a cell remains about the same (typically 12V). This means that the power density increases (amount of power in a given area) increasing the probability that the voltage will disturb the neighboring cells causing data corruption. Consequently, increasing the density of cells can be a dicey proposition, hence the lower limit.

Recently, IMFT (Intel-Micron Flash Technologies) LLC announced that it had begun sampling 2 bits per cell (MLC) NAND flash chips that have been manufactured using a 25nm process. This announcement is significant because of the increased density and how close the density is getting to the theoretical limit. Plus the fact that one of the best performing SSD drives is from Intel, one of the participating companies in the LLC, and we can see a fairly significant shift in technology.

IMFT is a joint project by Intel and Micro to develop new technologies around Flash storage (primarily NAND). The project started with production several years ago with a 72nm process. They then moved to a 50nm process in 2008 followed by a further reduction to a 34nm process in 2009. It is the 34nm process that current Intel SSDs utilize (Intel X25-M G2). The 34nm process produces a 4GB MLC NAND chip with a die size of 172 mm².The new 25nm process is targeted for a first product which is an 8GB MLC NAND flash chip with a die size of 167 mm². So going from 34nm to 25nm doubles the die density.

In addition to the doubling of die density the new chips will have some other changes. The current 34nm generation of chips have a page size of 4KB and 128 pages per block resulting in a block size of 512KB. The new chip will have a page size of 8KB and 256 pages per block. The means that the new block size is 8KB * 256 = 2,048KB (2MB). This change in block size can have a significant impact on performance.

Recall that a block is the smallest amount of storage that goes through an erase/write cycle when any single bit of the block is changed. For example, if any bit within the block is changed then the entire block has to first have the unchanged data copied from the block to cache and then the block is erased. Finally the updated information is merged with the cache data (unchanged data) and the entire block is written to the erased block. This process can take a great deal of time to complete and also uses a rewrite cycle for all of the cells in the block (remember that NAND cells have a limited number of rewrite cycles before they can no longer hold data).

The new 25nm chip will switch from 512KB blocks to 2MB blocks (2,048KB) increasing the amount of data that has to go through the read/erase/write cycle compared to the 34nm chips. To adjust to this change, Intel will have to make adjustments to the firmware to better handle the larger blocks. In addition, it is likely that Intel will have to at least double the cache size to accommodate the larger block sizes. It may have to increase the number of spare pages as well since a single bit change could cause a greater number of blocks to be tagged for updating. However, on the plus side, with larger pages, the controller can do much more optimization for writes including a much greater amount of write coalescing. But this too could increase the amount of cache needed.

It is projected that volume production of the new chips should happen in Q2 of this year (2010). This should result in new devices later in 2010. This article suggests that Intel could be shipping 600GB MLC based SSD devices in a 2.5″ form factor by the end of 2010.

Bigger and Badder

One of the puzzling things about SSDs, at least to me, is that so far they are all 2.5″ drives. Why hasn’t someone built a 3.5″ drive? Well, someone has. OCZ has launched a new drive series called Colossus. Their first product is a 3.5″ SSD drive with a 1TB capacity! You can also get the same drive but in lower capacities such as 256GB and 512GB.

The Colossus Series OCZSSD2-1CLS1T 3.5″ 1TB SATA II MLC SSD drive is available for purchase at just under $4,000. It is a 3.5″ SSD drive that uses a SATA II interface (3 Gbps max throughput) and has 256MB of on-board cache. OCZ states that the drive achieves about 220 MB/s of sustained read and write performance as well as 13,350 IOPS (4K random). The IOPS performance alone is equal to about 80-133 conventional hard drives.

There are several reviews of the drive. For example, PC Perspective has a nice review of the lower capacity Colossus drives where they tested the IOPS with a varying queue depth. The review used Windows (yuck) and they didn’t run the benchmarks several times as they should have but at least they examined the impact of queue depth on performance.

The price of the Colossus is steep but the performance appears to be pretty good and you do get a great deal of capacity unlike most other SSDs. However, if you are interested in pure performance there are some other SSD drives that plug into PCIe slots.

Unleash the Performance Beast

Fusion-IO realized that SSDs had tremendous performance potential but could be easily held back by the limitations of the SATA interface. Consequently, they developed a SSD drive that fits into a PCIe slot.

The SATA II interface is capable of 3 Gbps (375 MB/s) but a PCIe 2.0 x1 slot is capable of 500 MB/s and a PCIe 2.0 x4 slot is capable of 2 GB/s. It is very common to find x4 slots on almost all motherboards including servers and they have almost 10 times the theoretical performance of a SATA II interface (Note: the next generation SATA and SAS interfaces are capable of 6 Gbps, or 750 MB/s, and are just starting to appear – see this article). It was very perceptive of Fusion-IO to anticipate the pervasiveness of x4 PCIe slots and use them for very high throughput SSD devices.

Fusion-IO drives have remarkable levels of performance. The ioDrive Duo with 320GB of capacity is rated by Fusion-IO with a write bandwidth of 1.2 GB/s (32K packet size), a read bandwidth of 1.5 GB/s (32K packet size), and 185,022 IOPS (4K read packet size) or 129,699 (75/25 read/write mix with a 4K packet size). These levels of performance are quite remarkable for a drive that can fit into your desktop as well as your server.

For several years Fusion-IO has owned the PCIe SSD market. There are no other manufacturers but recently this situation has changed. Now OCZ offers PCIe drives that also have very good performance. They have created the OCZ Z-drive line of drives that fit into PCIe slots. The OCZ website states that they fit into a PCIe x8 slot. Assuming this means PCIe 1.0 then the theoretical performance is the same as the PCIe 2.0 x4 slot (2 GB/s). There are three drive models with difference characteristics as tabulated in Table 1.

Table 1 – OCZ PCIe Z-drives

The performance is very good in part because OCZ uses a 4-way internal RAID-0 configuration. The drives are not quite in the same speed class as Fusion-IO, but you can buy them from Newegg starting at $1,099 for the m84 with 256GB and going to $3,799 for the 1TB m84 drive (all prices were current as of the writing of this article. Please check the link for pricing and availability.). You can get a 1TB drive with remarkable performance for under $4,000! That’s the same price as the OCZ Colossus 3.5″ 1TB drive but with better performance.

Before departing from the subject of large drives with beastly performance, let me also point you to the announcement that a company named pureSilicon will be offering a 1TB SSD but in a 2.5″ form factor. No dates or pricing is available but it is likely to be shipping toward the end of this year.

Next: Combo Hard Drive and SSD

Combo Hard Drive and SSD

Sun Microsystems (before it became Oracle) started using SSD drives in combination with conventional hard drives. They could be used as high-speed cache devices and take advantage of the reliability of SSD drives relative to DRAM (i.e. no data lose when the power is lost). But this caching depends upon the file system supporting it. Current Linux file systems don’t have the ability to use any device as a “cache” so we can’t use SSDs as high-speed cache. However, an interesting recent product allows you to do this in hardware – sort of.

Silverstone has a product called HDDBOOST that mirrors a part of a hard drive onto an SSD drive without the involvement of the OS or the file system. The concept is pretty simple. You take an SSD drive and connect it and the hard drive to the motherboard SATA ports. Then there is a small controller that mirrors a portion of the hard drive to the SSD. If a data request can be fulfilled by the SSD then the response will be much, much faster than getting the data from the hard drive. These are read IO requests with any writes being handled by the hard drive bypassing the SSD.

Figure 1 below shows the inside of the HDDBOOST unit (from the Silverstone website) where the 2.5" SSD connects to the unit.

Figure 1: Inside of the HDDBOOST unit showing the location of the SSD connection.

It can accommodate most any 2.5″ SSD drive with a SATA connection but Silverstone recommends at least a 32GB drive. The SSD just slides into the connector inside the unit (no need for cables). It then screws into the unit itself to secure the drive. The unit with the drive is mounted in any open 3.5″ bay in the case.

Figure 2 below shows the front of the HDDBOOST unit (from the Silverstone website) that contains the connectors to the hard drive, the motherboard, and the power supply.

Figure 2: Front of the HDDBOOST unit showing the connectors for the hard drive and the motherboard.

Once you connect the SSD inside the unit, you then connect it to the hard drive via a SATA cable and connect the unit to the motherboard SATA connector via a second SATA cable. Finally, you connect the SATA power connector from the power supply to the unit. After connecting everything, you just fire up the system and, bingo, SSD caching.

When the unit is turned on, the controller mirrors as much of the hard drive as possible to the SSD. According to the manual on the Silverstone website, the first same size portion of the hard drive is mirrored to the SSD. For example, if the SSD is 32GB then the first 32GB of the hard drive is mirrored to the SSD. Synchronizing the mirror is done every time the server is rebooted. To get the best performance from the unit, Silverstone recommends that you reboot the server after a great deal of writing since the mirroring may not reflect all of the data on the hard drive that it should.

Also notice that the HDDBOOST unit can be plugged into a hard drive that has existing data (i.e. it doesn’t have to be a fresh drive). Also no special drivers are needed so it’s OS independent.

There are many questions around the unit but it’s only for sale in Japan at the moment. The mirroring process is a big question. Can the mirroring be “tweaked” so that it mirrors another portion of the hard drive? The second question is about the potential performance gains. Silverstone is claiming the performance can be boosted by up to 70% but this is untested in the open media at the moment. However, for $50-$55, it’s inexpensive enough for experimentation (minus the cost of the SSD of course).

See the Trend?

Sorting through the technology tea leaves can sometimes be challenging. However, from these three simple examples, it’s fairly obvious that SSDs are undergoing intense development.

The SSD industry is answering the cry about larger capacity SSD units. There are 1TB SSD drives available today helping to answer some of these critics. These drives are either 3.5″ drives or PCIe drives with the possibility of a 2.5″ drive later this year.

On the flip side, OCZ has entered the PCIe based SSD race. These drives are usually much faster than SATA 2.0 drives but are fairly pricey. Hopefully OCZ’s entry into the market will help drive down prices so that you don’t need a second mortgage for a PCIe SSD.

Also, we’re now seeing companies using SSDs in innovative ways. Silverstone’s HDDBOOST is an example of combining hard drives and SSDs for better performance (theoretically). It’s an inexpensive way to improve the read performance of hard drives. Look for more concepts like this one during 2010.

Finally, the joint Intel-Micron venture, IMFT LLC has developed the next generation of dense NAND chip. It effectively doubles the data density of current SSDs. Perhaps equally, if not more important, IMFT has found a way to maintain reliability while shrinking the chips from 34nm to 25nm. This could have a large impact on the next generation of SSD drives.

It’s an exciting time in the SSD world. Buckle up and geek out.