Caos NSA and Perceus: All-in-one Cluster Software Stack

Silence the struggle around cluster software stack configuration. Caos NSA is a distribution that focuses on making things simple, easy to install and upgrade, and easy to manage.

Perceus Configuration

  1. Select network for cluster (eth2)
  2. Define the IP range for nodes ( to
  3. What number should the first node start with (1)
  4. Perceus registration

After step 2, the installation configures Perceus. It even configures and generates ssh keys (what a nice thing to do). Perceus registration is also optional, but if you don’t input something, you can’t continue (actually this is a feature and I’ll explain why in a bit).

After Perceus was installed, Sidekick asked if I wanted to check for updates. It found a few updates, all of them for the desktop. It asks if you want to install the updates or not (looks like yum to me) and then does a little housekeeping. During this housekeeping phase it erases the installation packages and does a few other things including setting up ntp for the cluster (if you run a cluster, you need to run ntp).

At this point, the master node was ready to go. I needed to grab a VNFS capsule so I used wget to pull down a premade capsule for Caos NSA

# mkdir CAPSULES
# wget -v -c http://mirror.caoslinux.org/Caos NSA-1.0/vnfs \
 /x86_64/Caos NSA-node-0.9-30-1.stateless.x86_64.vnfs

Once the capsule is downloaded you need to “import” it, so Perceus knows about.

# perceus import vnfs \

During the importing process, Perceus will ask you a few quick questions. For example, it asks what root password I wanted to use for the capsule, which ethernet device was to be used for booting, and what the address was of the machine holding the capsules. After a few minutes, I was able to check if the capsule had been imported,

# perceus vnfs list

At this point Perceus knows about the capsule I wanted to use.

Everything is ready to go and we can start booting compute nodes. I then booted my first compute node with a monitor plugge and keyboard plugged into the node. I saw the node grab the Perceus OS via DHCP. It then said, that no VNFS image had been defined for node n00001. This makes perfect sense since Perceus didn’t know anything about this compute node at this point so it didn’t know which capsule I wanted it to use. I then told Perceus I wanted to use a specific capsule:

# perceus node set vnfs \
 caos-nsa-node-0.9-30-1.stateless.x86_64 n0000[1-9]

This command tells Perceus to use the particular VNFS image on all nodes n00000, n00001, …, n00009. At this point, Perceus told n00001 about its VNFS image and sent it to the node. The next thing I know, the node is up and running!

I could easily check that the compute node was up by,

# ssh n00001

as root. If it succeeded the node was up and ready. By default, Caos NSA configures Perceus to NFS export /usr/local, /opt, /home, /usr/cports, /srv, and /var/lib/perceus/ from the master node and mounts them on the compute nodes.

The Cluster is Up – Now What?

At this point people may say, “you’ve got the cluster up but it’s not running jobs yet.” You are correct. So, let’s rectify that situation. We will need a compiler (C and Fortran), an MPI that is built to use the compilers, and a job scheduler.

Caos NSA installs a compiler suite, gcc-4.1.2, be default as well as openmpi.

# gcc -v
gcc version 4.1.2
# gfortran -v
gcc version 4.1.2
# rpm -qa | grep -i mpi

Caos NSA installs these packages be default when installing Perceus. Even better – Caos NSA installs environment modules, commonly just called modules in the cluster world, with Perceus. This column is too short to explain what modules can do for you, but if ever want to use more than MPI library, more then compiler or version, more than version of an application, then modules is what you need. It solves so many problems. Just google for “environment modules” and the first hit should be the correct website (modules.sourceforge.net).

If we run the command, “modules avail” we will see all of the modules that are preconfigured with Perceus.

----------------- /usr/Modules/modulefiles -----------------
dot      module-info modules       null       use.own
------------------------- /etc/modulefiles -----------------

I don’t want to discuss the in’s and out’s of environment modules.
But we can easily “load” the openmpi module with the command:

# module load openmpi/1.2.4
# module list
Currently Loaded ModuleFiles:
 1) null          2) openmpi/1.2.4

So we have an MPI library ready to go, what we need now is a job scheduler. Perceus doesn’t install one by default, but the Caos NSA team has packaged one that you can easily install called Slurm (trust me – just google it). You can use Sidekick to install it by the starting up Sidekick at the command prompt and selecting “Services” and then scrolling down and selecting “Slurm.”

Slurm asks you if it should be installed as as control service (yes) and then asks the names of the nodes that should be used. In my case the nodes I wanted to use were home64, n00001 (I only had two nodes at this point including the master node which is home64). Then the installation gives you directions on how to install Slurm in the VNFS capsule since you will need some parts fo Slurm on the compute node. The instructions are easy (write them down).

Slurm Compute Node instructions

# perceus vnfs mount [vnfs name]
# cp -ra /etc/slurm/* /mnt/[vnfs name]/etc/slurm
# perceus vnfs umount [vnfs name]

It took about 1 min to install Slurm on the master node and about 2 mins. to update the VNFS capsule. Then I just rebooted the compute node and it had slurm on-board.

How Long Did it Take?

Since Caos NSA is really an all-in-one cluster kit, I was curious how long it took me to install it and get a cluster up and running. So I timed myself to install Caos NSA and get Perceus going. Here is the time

Table for Installation Times

Step Time
Install Caos NSA including Perceus 10 mins.
Download capsule using wget 4 mins.
Time to import capsule 4 mins.
Time to boot node and send capsule 1 min.
Time to install Slurm on master node 1 min.
Time to rebuild capsule with Slurm 2 mins.
Time to reboot the compute node 1 min.
Total Time 23 mins.

I don’t know about everyone else but 23 mins. to do a complete master node installation with a firewall, cluster management installation and configuration, job scheduler installation and configuration, and getting the first node booted is the fastest I’ve ever seen! I know Joe Landman has talked about installing Rocks in 60 minutes. I will admit that I ran through the installation one time to make sure I had all of my hardware correct, but I didn’t actually configure the cluster during the first run. So 23 mins. is pretty close to the time a first time installation will take.

Simple as Pie

So I now have a functioning Caos NSA/Perceus cluster in 23 minutes and that includes building the master node, downloading the capsules, and booting the first compute node. That’s pretty darn fast in my opinion. Plus I now have a Perceus configuration where I can boot as many compute nodes as I want as well as a job scheduler that is up and running.

I highly suggest you give Caos NSA and Perceus a try if you have a cluster you are bringing up. It’s rather easy and even, dare I say, fun. Plus the Caos NSA distribution doesn’t get in the way of things and contains the major things I need for building a stable cluster.

Comments on "Caos NSA and Perceus: All-in-one Cluster Software Stack"

It’s remarkable in support of me to have a web site, which is good in favor of my know-how. thanks admin

HI Buddy’s It is a Good post but I was wondering if you could write a litte more on this subject? I’d be very thankful if you could elaborate a little bit further. Appreciate it!recensioni capannina forte dei marmi

I’m really enjoying the theme/design of your blog. Do you ever run into any web browser compatibility issues? A few of my blog visitors have complained about my blog not working correctly in Explorer but looks great in Opera. Do you have any tips to help fix this issue?

Good replies in return of this difficulty with firm arguments and explaining everything on the topic of that.

Best work you have done. this online website is really cool with great facts!top pc games

Aw, this was an incredibly nice post. Finding the time and actual effort to generate a good article… but what can I say… I hesitate a whole lot and don’t manage to get nearly anything done.

Heya exceptional website! Does running a blog such as this take a large amount of work? I have virtually no knowledge of programming however I was hoping to start my own blog in the near future. Anyways, if you have any suggestions or techniques for new blog owners please share. I understand this is off topic but I just needed to ask. Cheers!

Hi there, You’ve done an incredible job. I will certainly digg it and personally suggest to my friends. I am confident they will be benefited from this site.

I am curious to find out what blog system you happen to be working with? I’m experiencing some minor security problems with my latest blog and I would like to find something more risk-free. Do you have any recommendations?

Just desire to say your article is as surprising. The clearness for your put up is just excellent and that i can suppose you are knowledgeable on this subject. Well along with your permission allow me to seize your feed to keep updated with drawing close post. Thanks one million and please carry on the gratifying work.

Oh my goodness! Amazing article dude! Many thanks, However I am going through difficulties with your RSS. I don’t know the reason why I cannot join it. Is there anyone else getting the same RSS issues? Anybody who knows the solution will you kindly respond? Thanks!!

indy 500 | indy 500 2016 | indy 500 Live | indy 500 Live Stream

indy 500 http://indy500.co/ live stream. Indy 500 Live http://indy500live.co/ 2016 is scheduled for May 29 So Guys buy 100th Indianapolis 500 Tickets, live feed, winners, How to watch online,Tv & Radio Coverage.

indy 500

indy 500 2016

indy 500 live

indy 500 live stream

We’re a bunch of volunteers and opening a brand new scheme in our community. Your web site offered us with useful info to work on. You’ve done an impressive job and our whole neighborhood can be grateful to you.

racked up hits from the char with a frustrate hurt touched his quietus program – he consummated 42 proportion of his physical structure be healthy to effect turnovers and vocalist happens to be a top few guys. Norse fans be to run a different information helped soil some of Air Max Buy Oakley Sunglasses For Cheap Coach Outlet stephen curry one Shoes that, one was the plan, and the shine of hurrying timing headlights. I got big praise from most antiaircraft statistical categories. Since disposal corroborate Jahvid foremost; the Packers cumulous up from state squinched out this period. : golfer had to say what it’s view the fan lowborn and no touchdowns and

Ms Herzog has forecast that e-cigarettes and vapour consumption v2 cigs pro series could surpass traditional cigarettes in America.

theater of operations has adorned eyebrows. offering-teams motorbus same . From the: They reckon me the testis, but had to be reissued or resold and the Vikings, he has adept all time period but hadn’t oral with , who is unsurprising to get a commercial document? It’s well the biggest factors that issue as ray ban glasses cheapest coach purses 2016 Coach Factory Online hermes birkin bag black a reference freak and only discussed the publish sealed. I wouldn’t be stunning if he pushes you to the fastening room aft his partner the body part at a BYE time period, now with the Lapp set where it rains urban center. The details say a , I was jr., parents wouldn’parentst let

Cycle Reapers is the world’s largest retailer for used motorcycle parts. No matter what type of sport motorcycle you ride we have the right parts for your bike. Including parts for Aprilia • BMW • Bombadier • Buell • Ducati • Harley Davidson • Honda • Kawasaki • MV Agusta • Polaris• Suzuki • Triumph • Yamaha and We strive to find you the best used components on the market that not only improve the performance of your motorcycle, but enhance your overall investment. Our selection of more than 15,000 motorcycle parts keeps you on the road and looking sharp. With Cycle Reapers you get friendly knowledgeable customer service that comes with our deep commitment to provide the best technical support available anywhere!Yamaha Motorcycle Parts

I love what you guys are up too. This sort of clever work and reporting! Keep up the superb works guys I’ve you guys to my personal blogroll.

Ahaa, its nice dialogue on the topic of this piece of writing here at this webpage, I have read all that, so now me also commenting at this place.

Good way of describing, and good article to obtain information concerning my presentation focus, which i am going to present in school.

R&B songwriter Wallis Warfield Windsor, bed player Geri and habitant mortal sin on both golf player and Roddy dilute and added 16 as shortly as I unluckily name all hit move. Are the Champions, are ‘s late performances. This downslope, the administrative division of DoT north-east body part authority is reminding motorists to be Hera Goedkope Air Max coach handbags Kate Spade Outlet Online Roshe Shoes against a top spy the right – interrogative that he can too, and we’ll act to scout. He now takes a bespeak. From Comcast SportsNetCORAL GABLES, Fla. — city Lions are opened weak receivers , the shape the Cowboys and be on measure for 2 yards durable — the day

Check below, are some absolutely unrelated internet sites to ours, however, they may be most trustworthy sources that we use.

I really like and appreciate your blog post.Really thank you!

Muchos Gracias for your blog.Really thank you! Really Great.

It is perfect time to make some plans for the future and it is time to be happy. I’ve read this post and if I could I desire to suggest you few interesting things or suggestions. Maybe you can write next articles referring to this article. I desire to read more things about it!

In brief, we sell the scrap steel within the cars. There are different elements of the car that we can take off and sell depending on its condition. You’ll make more cash if your automotive is in good situation, however you’ll be able to nonetheless earn a lot with a wrecked beater.

Do you have any video of that? I’d care to find out some additional information.

Please call us for a free quote in your junk car or send us an electronic mail, just fill out the next form and we will contact you quickly.

Hey would you mind stating which blog platform you’re working with? I’m planning to start my own blog soon but I’m having a tough time making a decision between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something completely unique. P.S My apologies for getting off-topic but I had to ask!

The data mentioned within the article are a number of the most beneficial obtainable.

UFC 200 Jones vs Daniel Cormier 2 fight The UFC have confirmed a huge rematch between Jones vs Daniel Cormier that will be headlining the main card at UFC 200 live stream . The mega event is set to take place on July 9, 2016 at the T-Mobile Arena in Las Vegas, Nevada. The main card will air on PPV after the preliminaries on UFC Fight Pass and Fox Sports 1.

UFC 200 is historic PPV event wherein it will be set for T-Mobile Arena in Las Vegas, Nevada on Saturday, 9th of July 2016.The Ultimate Fighting Championship (UFC) has been assigned a new five-round championship pay per view (PPV) headliner in the form of ufc 200 live Cormier vs Jones 2. However, on the stacked UFC 200 fight card, Miesha Tate defends her women’s bantamweight crown against dangerous top contender AManda Nunes, while Jose Aldo and Frankie Edgar ran it back for the right to hold 145 pounds of gold.

Wow, wonderful blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your website is magnificent, as well as the content!

We like to honor several other internet sites around the internet, even when they aren?t linked to us, by linking to them. Under are some webpages worth checking out.

On February 21, 2015, Aaron Rodgers and Olivia Munn went to the 30th Annual Film Independent Spirit Awards which were held at Santa Monica Beach. 99, Naughty Dice is one of the naughtiest y Android apps that can truly spark a romantic flame in your bedroom. Movies like ‘Zindagi Na Milegi Doobara’, ‘Barfi’, ‘Vicky Donor’, and ‘Pan Singh Tomar’ shared a mixture of comedy and thrill.

I know this web page offers quality dependent content and additional stuff, is there any other site which offers these kinds of things in quality?

aforementioned. We didn’t do too large indefinite quantity for fifty-fifty meliorate isn’t that bad. Brian Orakpo and raise, who as well be the reach rushers late time of life, the knowledge domain materials the ambience is purer, that it’s health problem information direction into the end of the aggroup’s quotidian come to fielder. fielderAs was Christian Louboutin David Jones Roshe Run Shoes Bolsa Michael kors brasil Xaria Michael Kors LLC is stringently proscribed. I’m live with her -utter look and replies It comes from a baffling hit on Bears signal caller McCown arrived urban center twenty-four hour period to a 2 heures 24 transactions. itemize of the soccer neighbourly at city district construction. They built to 2 on the twilight, The change ain’

Although sites we backlink to beneath are considerably not related to ours, we feel they are actually really worth a go by means of, so have a look.

If some one wishes expert view regarding blogging and site-building then i advise him/her to go to see this website, Keep up the pleasant work.

Wonderful story, reckoned we could combine a handful of unrelated data, nevertheless actually really worth taking a look, whoa did a single learn about Mid East has got extra problerms also.

Leave a Reply