From Russia with Love: POHMELFS – A New Distributed Storage Solution

There is a new file distributed file system in the staging area of the 2.6.30 kernel called POHMELFS. Sporting better performance than classic NFS, it's definitely worth a look.

POHMELFS Performance

There is a very good blog that discusses how to build and install POHMELFS including the user space tools. You can also read the kernel documentation around POHMELFS to learn about the mount options.

Assuming that you have POHMELFS installed and configured then the first thing one wants to do is run tests. On the POHMELFS website there is a blog post that presents some benchmark comparisons. The configuration is a dual-socket server node with only 1GB of memory connected via a GigE network to a dual-socket client node. The benchmark ran IOzone, bonnie++, and dbench.

From IOzone, it was found that POHMELFS has very good read, write, reread, and rewrite performance. However, it does not have good random read performance. From Bonnie++ the performance was better than NFS but it had some small trouble computing the object creation/removal times since the local data writeback cache of POHMELFS is very fast compared to NFS. However, dbench would not run very well at that time because the rename operation is synchronous and ran rather slow. However, Evgeniy was going to work on that aspect of POHMELFS.

From these simple benchmarks it can be seen that the performance of POHMELFS is very interesting and compelling over the benchmarks tested. It is a very interesting distributed file system that should be tested if more performance than NFS is needed. The choice of replacing a classic NFS configuration with POHMELFS configuration is up to you but it does warrant investigation because it can has good performance and can solve some problems with NFS.

POHMELFS Roadmap – The Elliptics Network

At this time POHMELFS is very much a NFS-like file system. Evgeniy also discusses how he doesn’t believe that POHMELFS in it’s present form, will replace NFS. He argues that it is not a distributed file system. But he also states that he will be leaving it in the staging directory while he ports it to use the Elliptics Network.

The Elliptics Network is a fault tolerant distributed hash table object storage system (lots of concepts in there). Using this as a basis for a distributed parallel file system can create a resilient parallel distributed file system. In particular, a distributed hash table design. Recall that a hash table is a data structure that uses a Hash function that maps keys or identifiers to their associated values. For example, you can use an identifier such as a person’s name (the key) and get the person’s address. It is a very popular data structure that can be used for a huge number of tasks and is the essential part of many languages such as Perl and Python. Below, in Figure 2, is an image from wikipedia that illustrates a distributed hash table

Figure 2 - Distributed Hash Table (from Wikipedia)
Figure 2 – Distributed Hash Table (from Wikipedia)

Distributed hash tables have been used in peer-to-peer network systems such as Napster or Gnutella. For file systems they provide some immediate benefits.

A Distributed Hash Table takes the basic hash table and decentralizes the hash table. It allows any node (server) to perform a hash lookup (key, value) and retrieve the data. All of the participating nodes share the task of maintaining the key and value mappings. But the design of the distributed hash table allows nodes to join, leave, and fail with minimal disruption. For large distributed file systems, distributed hash tables can be a good starting point for a file system design. This is exactly what Evgeniy has planned for POHMELFS – the ability to add, remove, or have servers fail that are part of the file system with little or no impact to the file system. In addition, using a distributed hash table means that there doesn’t have to be a single metadata server (one of the limitations of NFS).

POHMELFS will be ported to use the Elliptics Network as the underlying data structure while keeping the client interface the same. Work is just beginning on this port.


POHMELFS is an object based file system that has been developed to reduce or eliminate many of the NFS bottlenecks (with the exception of NFS v4.1). It is a parallel and distributed file system that allows data servers to be removed, added, or even fail while the file system is functioning without much of an impact on performance. Moreover, it allows data to be distributed across the IO nodes and also balance read operations across the IO servers. Coupled with this is a local cache capability for both metadata and data that maintains coherency across the servers.

Initial performance results of POHMELFS versus NFS show some promise. The results are a bit old, but they do indicate the POHMELFS can have very good performance.

But POHMELFS isn’t done. While the current version is usable and ready for testing it is still in the “staging” area of the kernel (i.e. not ready for prime time). In addition the developers are porting it to use a distributed hash table called Elliptics Network. There are lots of features in Elliptics network that work well for a distributed file system as previously discussed.

While POHMELFS might not be ready to replace NFS, it does have some benefits that can prove very useful in some situations. However, be warned that POHMELFS is very much a work in progress. It is in the “staging” area of kernel drivers and the developer has announced that POHMELFS will be ported to use a distributed has table.

Comments on "From Russia with Love: POHMELFS – A New Distributed Storage Solution"

If you have wholesome kidneys, I extremely recommend that you supplement with creatine.

Here is my web site – female caloric intake for weight loss

Im obliged for the article. Cool.

Do you have a spam issue on this blog; I also am a blogger, and I was wondering your situation; we have developed some nice procedures and
we are looking to trade techniques with other folks, why not shoot me an e-mail if interested.

Highly energetic article, I loved that bit. Will there be a part

Feel free to visit my site – hibu Financial News

Great article! This is the kind of information that should be shared around the web.
Disgrace on Google for now not positioning this put up upper!
Come on over and visit my site . Thanks =)

Wonderful goods from you, man. I’ve take note your stuff previous to and you are just too wonderful.
I really like what you have bought here, certainly like what you’re stating and the way in which you
assert it. You make it entertaining and you still care for to keep it smart.
I can not wait to learn much more from you. That is actually a wonderful site.

I simply want to say I am just new to weblog and actually enjoyed this website. More than likely I’m want to bookmark your blog . You certainly have impressive article content. Appreciate it for sharing your webpage.

Hi there it’s me, I am also visiting this web site regularly,
this web page is genuinely nice and the users are genuinely sharing
fastidious thoughts.

Every as soon as inside a although we choose blogs that we study. Listed beneath would be the most current websites that we choose.

Leave a Reply