An HPC Field Trip

When you work at home in your basement, “getting out” takes on a whole new meaning. A highlight of my day is heading down to some local coffee joint and grabbing a warm cup of inspiration (such as the one that is in front of me now). Of course I could, fire up my own coffee pot, but I like to get out for a bit. Sometimes on my little excursions, I even stop at the grocery store and pick up a few necessities for the family.

Given the level of excitement in my daily life, you can understand when I get a flier for the annual Linux on Wall Street meeting I get all excited. The meeting name has been changed to “2010 High Performance Computing Linux Financial Markets,” but the vendors and the customers seem to be the same. There is also a fall meeting called 2010 High Performance Computing Financial Markets, which seems to be the same as the Linux meeting, except Microsoft has a table. Not that it matters to me. All the more reason to take the two hour bus ride into the big apple and wade around in some HPC. Of course, nothing compares to the Supercomputing (SC) show in November of each year. I consider SC less of a show and more of a happening where the ultimate geek city is built once a year.

I have attended most of the Wall Street shows and noticed that the financial sector seems to have bounced back from its recent hiccup. On this very nice April day, I enjoyed the sights and sounds on my walk from the bus terminal to the Roosevelt Hotel. Upon arrival, I grabbed my press badge and almost walked right into Matthijs van Leeuwen from Bright Computing. From what I had gathered, Bright Computing provides cluster management software. I was not sure what their “edge” was because there is certainly no shortage of cluster solutions out there.

Bright has an interesting and certainly obvious solution to what some consider a problem with clusters. If you peak inside most clusters you find a broad array of software used to do everything from provisioning, to monitoring, to management. There are many choices and tools available for these tasks. Each tool pretty much works independently and is not aware the other tools running on the system. In many cases, there are often several redundant monitoring daemons running on each node. For the most part, the daemons monitor the same information and then send it to a central server for logging or reporting.

Bright Computing addresses this situation by providing a unified set of tools and a framework to manage a cluster. Instead of trying to weave together a raft of open source solutions, Bright Computing has started with a clean slate and created cluster provisioning, monitoring, and management from the ground up. They work with the standard Linux distributions and allow for your choice of scheduler. It is basically a turn-key supportable solution for clusters. That said, the Bright solution is closed source so it may not be appealing to the rugged HPC pioneers who like access to the plumbing. You can find out more at their website.

Later in the day, I had brief chat with Sujal Das, Senior Director, Product Management at Mellanox. Sajal explained how the use of InfiniBand is enhancing Cloud services. While Cloud and HPC are not yet a good mix, Mellanox has been deploying InfiniBand into Cloud based systems with impressive results. Improved I/O and thus the promise of better Service Level Agreements (SLA) is attractive to Cloud providers. You can read more from the Mellanox Press Release.

In years past, I have attended some of the talks at these meetings. Like many conferences my interest level ranged from high to “I think I’ll go get some coffee and check my email.” Many of the talks are actually vendor panels that address a specific topic (or product line). I sometimes drink a lot of coffee at these events. This year I was pleasantly surprised to see Chuck Seitz from Myricom giving a talk entitled “High Speed Trading Can Leverage HPC-Cluster Techniques.” I have known Chuck for quite some time and he usually has something interesting to say. I was not disappointed and actually participated in his talk!

The first part of Chuck’s presentation was a basic background on HPC clusters. As we both have put our time in the HPC market, Chuck asked my opinion on a few topics and even mentioned the free HPC for Dummies mini-book I had written for Sun and AMD. For those that do not know, Myricom is a leader in high performance interconnects for HPC. Their principle product line is now high performance 10G Ethernet. They also provide the MX high performance kernel by-pass library used by many MPI packages.

Chuck likes to talks about the two networking technologies used in high performance computing, “There is Ethernet and Ethernot.” Myricom is clearly an Ethernet company and have leveraged their expertise in high performance networking to create the Datagram Bypass Layer (DBL) library. DBL is an optional, user-level, software interface (library) for accelerating applications that depend on UDP/IP communication. DBL was developed principally for financial applications, but can be very useful in other areas as well. Conventional UDP communications pass through the host operating system, which allows the full set of IP networking services, but at the cost of message latency (messages must traverse the protocol stack and be copied to/from system buffers). DBL takes advantage of kernel-bypass techniques to send and receive UDP frames directly resulting in a user-level latency of about 2.5 μs. DBL is not intended to replace standard UDP communication, but rather DBL will improve the throughput, response time, and transaction rates of user applications that use the DBL services. You can find more on DBL from this short write-up. The talk concluded with Chuck’s description of a TCP/IP by-pass library that is currently under development. It is nice to see the lessons and successes from HPC grow into the mainstream.

For the rest of the day, I talked with some other HPC mavens and then headed down to the lounge for a way to expensive New York beer (or two) and further discussion. By this time, I had plenty of HPC and enough material to read on my way home back to the basement. I need to get out more often.