Drizzle: Rethinking the MySQL Database Kernel

If you visit the Drizzle project homepage, it explains:

Drizzle is building a database optimized for cloud and [network] applications. [Drizzle] is being designed for massive concurrency on modern multi-cpu/core architectures. The code is originally derived from MySQL.

In other words, Drizzle is a re-thought and re-worked version of the MySQL kernel designed specifically for high-performance, high-concurrency environments. The goal is to make Drizzle an extensible database kernel that runs extremely fast on emerging hardware.

Much like Linux is a fast kernel that can be extended with loadable modules (notably filesystems and device drivers, among others), Drizzle explicitly says “no” to building in some features, instead pushing them out of the core and into plug-ins. This separation allows Drizzle to focus on only the most essential features of a modern database kernel. In fact, it’s almost a become a joke on the Drizzle mailing list that someone will say “Make it a plugin!” when anyone questions removing functionality from the server.

At this point you may be wondering who would be crazy enough to think that MySQL, known for its speed and agility, needs to be radically re-worked. After all, it’s been the dominant open source relational database for the better part of a decade now.

The Drizzle project is headed by Brian Aker, the former director of architecture for MySQL. Aker now works in the Office of the Chief Technology Officer at Sun (soon to be Oracle) and heads up a team of developers working on Drizzle and a few other projects (see one of Aker’s early posts on Drizzle). Some of those developers are Sun employees (Eric Day, Jay Pipes, Monty Taylor, Mark Atwood, and others), while others are either volunteers or contributing to Drizzle on behalf of their employer. Toru Maesaka of Mixi is one such developer.

A lot of the Drizzle work is driven by significant changes in hardware that have occurred since widespread adoption of MySQL began. Before discussing what Drizzle is and is not, it’s important to understand a bit about the last 5-7 years of evolution in hardware and systems and the challenges those presented to MySQL users, especially those who were constantly pushing the envelope of what’s possible.

Moore’s Law Turned Sideways

The most significant change in recent hardware history is that Moore’s Law took a surprising turn. For years, developers have been able to depend on the clock speed of CPUs doubling roughly every 18 months. In other words, the code you ran in early 2001 would be more than twice as fast if you simply upgraded the CPU in your server in late 2002. Why “more than” twice as fast? Not only did clock speeds increase, changes in the architecture of CPUs made them more efficient, too: larger caches, more registers, better instruction pipelines, and so on.

This doubling trend went on for so long that Moore’s Law was practically law and something destined to continue for the foreseeable future. But then something strange happened a couple years ago. Instead of seeing 4 GHz CPUs appear to replace 3.2 GHz CPUs, Intel and AMD started pushing multi-core CPUs that ran at slower clock speeds. Effectively, the megahertz and gigahertz game had come to an end. Instead of pure “vertical” scaling, Moore’s Law had been redirected along a different axis: horizontal scaling.

In a sense this isn’t new. Symmetric multiprocessing (SMP) has been around for years now in the form of motherboards with two or four physically separate CPUs. But having multiple CPU cores on the same physical chip was a new approach. Intel previewed the future with the introduction of Hyper-Threadingon some Pentium 4 CPUs. The technology allowed a single CPU to run two execution threads in parallel. It’s not the same as having two physical cores on the same chip, but it was a nice boost at the time.

With Moore’s Law overturned, the new road to efficiency and performance is to design for concurrency and parallelism. Any problem that can be split into sub-problems and executed in parallel fits this new world very well. As you’ll see momentarily, there are places where MySQL is able to exploit multi-core well, but others in which it was of no help at all.

RAID and RAM

As with all things hardware, the cost of running substantial RAID arrays (6-12 disks) with battery-backed write caches has dropped substantially. That doesn’t mean that disk I/O isn’t still the single biggest bottleneck in large databases—it is. Any time your data size grows substantially beyond what can fit in RAM, I/O bottlenecks appear. But today’s “commodity” hardware lets you push that barrier out quite a long way.

Thanks to commodity 64-bit CPUs, it’s not hard to find a server than can be loaded with 64 GB or even 128 GB of memory. That alone is sufficient to keep a very large percentage of MySQL deployments from having to worry about going to disk that often. But what about those really big deployments?

The transfer rates of modern disks and their controllers is measured hundreds of megabytes per second. But that’s assuming a best case scenario: linear (sequential) reads or writes. The real killer is random disk I/O. Any time the disk head needs to move (seek), you incur a delay of several milliseconds. Event the fastest hard disks available claim average seek times in the 4 ms range. Expecting more than about 200 seeks per second is unrealistic. In fact, the rule of thumb for years has been not to expect more than about 100 I/O operations (iops) from a single spindle.

Having larger disk arrays means being able to perform a greater number of iops per second: typically something like (N/2)*100 where N is the number of disks. You must divide by two because of mirroring. In a typical setup of 12 disks, there are 6 mirrored pairs (RAID-1) and stripe (RAID-0) data across them. This is commonly referred to as RAID 1+0.

Using this model, the only way to get another 200 iops out of your array (without sacrificing data integrity) is to add two or four more disks. Like Moore’s Law, this had been true for a very long time. But just like Moore’s Law, the storage game has recently changed in a most unexpected way.