The State of Oracle/Sun Grid Engine

Things change. Indeed, high tech is often a sea of change. For some open source software is a lifeboat in the storms of change. While everything else can be bought, sold, or demolished, your investment in open software is still yours. No where is this more important than in HPC where change is a way of life. Prior to open source, the ability to use hardware often depended on support contracts from vendors. If these companies sank, so did the support and keeping with the ocean metaphor, your prized HPC system has now become an expensive boat anchor.

Open source (and commodity hardware) changed this situation. Today, vendors can vanish, but it is still possible to update and modify the software. While the preceding philosophical rant may seem like the same old song and dance, there an open source scenario playing out in the HPC sector right now. Almost everyone is aware Sun was purchased by Oracle. It also appears that Oracle is not really interested in HPC. Once key piece of HPC software that Sun provided was Grid Engine. Grid Engine is a distributed resource scheduler that allows job management across clusters and workstations. As of version 6.2u5 Grid Engine was licensed under the OSI approved Sun Industry Standards Source License (SISSL). As I understand, Grid Engine is the only remaining piece of Sun software that uses that license.

Let’s take a look at the good news. According to DanT’s Grid Blog Oracle has plans for Grid Engine. As Dan mentions, Grid Engine does not compete with any Oracle product and like other resource managers has applications outside of HPC, which probably means Cloud based solutions.

Now let’s take a look at the, uhm, other news. As I mentioned, version 6.2u5 was licensed under SISSL and is in wide use. As a mater of fact, I have been using Grid Engine for over ten years. In addition to being very good at resource management, it has a very active support community. As far as open software projects go, it has a healthy Eco-system. Sun had been selling a supported version of Grid Engine as an official “product.” They also have been giving away “courtesy binaries” to the community. As I understand it the official product is slightly more polished with some added features than the courtesy binaries. As an aside, if there was ever a product (open or closed) where support is key it is distributed resource managers.

Back in June of 2009, Sun made a change in the evaluation license for the “Sun Grid Engine Product” starting with version 6.2u3. The license is for a 90 day evaluation. At the same time,
both source and courtesy binaries were furnished on the http://gridengine.sunsource.net site which is not the same as the product site: http://sun.com/sge.

There has been some concern due to this article, Oracle nixes free Sun GridEngine, goes 90-day trial, which as you read the title sounds pretty bad. The article talks about the new version 6.2u6 and the 90 day evaluation in the EULA that has been there since version 6.2u3. To be clear, as of right now, both the source and the open source courtesy binaries are freely available for 6.2u5.

The concern seems to center on the recent release of the 6.2u6 product binaries, but no coresponding update to the open source site (r the fact that no patches have been commited to the open source version since 6.2u5 release). I queried the Beowulf Mailing list and I think Joshua Baker-LePain said it best:

“To clarify a few minor points here (again, as *I* understand it), the eval
period is actually 90 days and does *not* apply to the “courtesy
binaries”, which are available up to and including 6.2u5 and are missing a
couple of modules. The concerns on the SGE list seem to have arisen
because a) there are no courtesy binaries for 6.2u6, b) no post-6.2u5
patches have made it to the source repository, and c) there has been
little to no word from Oracle as to future plans for source code access
and courtesy binaries.”

There is no doubt some things are changing, but it is not clear if these were in the works at Sun or are something Oracle is orchestrating. You can read some interesting threads on the SGE discussion list or on the Rocks List. This last post by Andy (I believe this is Andy Schwierskott who is on the Grid Engine team) on the SGE list seems to sum up the situation:

“the open source based courtesy binaries of Grid Engine 6.2u5 come with a free, unlimited license.

The Oracle/Sun Grid Engine 6.2 Update 6 release from http://sun.com/sge come with a 90 day time limited eval license only.

Things may not be as dire as they may seem at this point. The question on everyone’s mind is “What about the 6.2 Update 6 sources and courtesy binaries?” The has not been answered by Oracle and I think this is the concern of the community. It is possible that the open source versions may trail the product version by a few months as has been done with other projects. Or, Oracle is figuring out what to do with whole open/closed Grid Engine model. Integrating the two companies takes time and before we condem Oracle, it may be wise to give them a little room to do the right thing. Remember, it was Sun who introduced the evaluation license.

As I see it, there are two things that can happen. First, Oracle can continue to support the open source HPC community and enjoy the benefits that it imparts to the commercial version. A healthy open source Eco-system with lots of smart people helping you create/refine/improve your product while at the same time increasing “mind share” is the envy of any marketing department (and it does not come cheap). My message to the marketing crew, before you get all caught up in the lost sales numbers because of all the free as in beer users, think about how much it would cost to position your product in the top 3 resource schedulers. Also remember, many of those free as in beer users are also your beta testers, focus groups, and consultants that would normally cost you more money. Many of them are, after all, the rocket scientists everyone talks about.

The other option is that the project will fork. This option is very real and the recent Open Grid Scheduler project may kick into high gear in the near future should Oracle drop the ball as it were. The only thing, in my opinion, preventing the fork is if Oracle continues Sun’s embrace of the open source SGE community.

Let’s end on some more good news. If you are a open Grid Engine user, your are protected by the “Open Source insurance plan.” Should Oracle not embrace the open Grid Engine model and community, then the Open Grid Scheduler fork will happen. Of this I am sure. This is where the free as in speech part takes over and philosophical ideas turn into action.