Slap a Patch on It and Call it Good

There’s nothing more frustrating than phoning in a support incident and the voice on the other end of the line stabs you in the heart with the most often asked question in the IT business, “Have you downloaded and installed the latest patches for your distribution?” She might as well have taken my first-born to the local volcano for sacrifice or asked me if I’ve read the documentation. Of course, the answer is, “No, ma’am, I haven’t. Shall I do that now?” The second and more painful stab comes from her answer. “Yes, please.” Just take my first-born, he’ll never know the difference and it’s easier than paging through documentation.

Patching is a time-consuming and thankless task. The overlords don’t understand the number of hours System Administrators spend applying patches, correcting problems that the patches cause, and applying patches to patches that didn’t go so well. There’s no single right answer to patch madness but there are some guidelines that should govern us all.

The Basics

Knowing how to patch your system is the first step in keeping those insistent phone support types happy. And, as a System Administrator, it also provides some level of job security. There’s nothing like the compromise of an unpatched system that gains you C-level notoriety and a place in the unemployment line. Learn a few simple commands to protect you and your system. Updating is easy and should be performed regularly. And, remember, since you’re using Linux, most updates require no system restart.

For those of you who use or manage systems with graphical desktops installed, your task is relatively easy and you have graphical applications that notify you of system updates and they’re simple to use: A few mouse clicks and you’re done. But, what about those who manage ‘server’ systems that have no GUI or desktop manager installed? Some distributions alert command line users to available updates upon login. Whatever your particular case, you need to learn the fine art of patch management. Let’s get started with that exercise.

Note: The systems used for this article are CentOS 5.5 (rpm-based) and Ubuntu 10.10 (deb-based).

Updating Graphically

Click on the Notification Icon, enter your root password to open the Update Manager application and start the update process. See Figure 1.

Figure 1: Notification of New System Updates

Ordinarily, the Update Manager will open, you’ll deselect any applications that you don’t want to update, if any, click Install Updates and continue the process. Sometimes you receive the message shown in Figure 2. This message explains that not all of the installed packages can be updated as is. Unless you have services or appliations that you want to preserve, select Smart Upgrade to continue.

Figure 2: Graphical Update Manager with Warning

The system decides which packages to safely update, resolves dependencies, and returns you to the Update Manager. Select Install Updates. When finished updating, close the Update Manager application. If instructed to reboot your system, do so.

Command Line Updates

login as: khess
[email protected]'s password:
Linux vlad 2.6.35-24-generic-pae #42-Ubuntu SMP Thu Dec 2 03:21:31 UTC 2010 i686 GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

14 packages can be updated.
4 updates are security updates.

Last login: Sun Feb 10 22:31:47 2011 from xenalive
khess@vlad:~$

This message lets you know that there are updates available and you don’t have to check for yourself. So, like any good Ubuntuer, take care of those updates without hesitation. The Debian-style update procedure uses apt-get to search repositories and return

$ sudo apt-get update

$ sudo apt-get
...
133 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 66.2MB/192MB of archives.
After this operation, 1,094kB of additional disk space will be used.
Do you want to continue [Y/n]? y
...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
Processing triggers for python-support ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-2.6.35-24-generic-pae

And, that’s all there is to performing a system update. If your update includes a new kernel, you should reboot the system to launch the new kernel. Some distributions will prompt you to reboot after you install a new kernel. Yes, there are ways around rebooting after installing a new kernel but rebooting doesn’t cost anything and takes very little time.

For users of RedHat-based systems, you’ll use yum to check for available updates.

$ sudo yum check-update
...
sox.i386                        12.18.1-1.el5_5.1           updates
sudo.i386                       1.7.2p1-9.el5_5             updates
tcsh.i386                       6.14-17.el5_5.2             updates
tomcat5-jsp-2.0-api.i386        5.5.23-0jpp.11.el5_5        updates
tomcat5-servlet-2.4-api.i386    5.5.23-0jpp.11.el5_5        updates
xorg-x11-drv-ati.i386           6.6.3-3.27.el5_5.1          updates
xorg-x11-server-Xnest.i386      1.1.1-48.76.el5_5.2         updates
xorg-x11-server-Xorg.i386       1.1.1-48.76.el5_5.2         updates
xulrunner.i386                  1.9.2.13-3.el5              updates

This list tells you that there are several updates that require your attention. To update your system, use the following commands.

$ sudo yum update
...
Transaction Summary
=======================================================================================================
Install       1 Package(s)
Upgrade      70 Package(s)
Remove        1 Package(s)
Reinstall     0 Package(s)
Downgrade     0 Package(s)

Total size: 321 M
Total download size: 76 M
Is this ok [y/N]:y
...
sox.i386 0:12.18.1-1.el5_5.1
sudo.i386 0:1.7.2p1-9.el5_5
tcsh.i386 0:6.14-17.el5_5.2
tomcat5-jsp-2.0-api.i386 0:5.5.23-0jpp.11.el5_5
tomcat5-servlet-2.4-api.i386 0:5.5.23-0jpp.11.el5_5
xorg-x11-drv-ati.i386 0:6.6.3-3.27.el5_5.1
xorg-x11-server-Xnest.i386 0:1.1.1-48.76.el5_5.2
xorg-x11-server-Xorg.i386 0:1.1.1-48.76.el5_5.2
xulrunner.i386 0:1.9.2.13-3.el5

Complete!

Yes, you’re seeing that last bit correctly. When you’re a Red Hat or Red Hat derivative user, you’re blessed with an exclamation point to punctuate the completion of your patching.

You’ll find that once you’re in the patch groove, it becomes second nature to schedule and patch on a regular basis. But, don’t jump into the latest patches and fixes in haphazard fashion or by automating them with a scheduled patching script. The problem with unattended patching is that you have no control over them. Controlled patching is important. What if you install a patch that causes a kernel panic and it results in a day or more of downtime for your system?

Develop a patch plan and stick to it. Simply put, you should patch on a quarterly basis. Any more often than quarterly and you run the risk of installing non-regression tested patches. Any less often and the risks should be obvious: You risk losing control of your system to hackers or system panics caused by unstable software. Security patches should be installed as they appear to prevent falling prey to attacks. It really isn’t difficult to develop your own patch management plan. So, today you’ve learned a new acronym: PTFM, where P=Patch and M=Machine. For further reference, see RTFM.

And, the next time you call tech support, you can respond to the “Have you downloaded and installed the latest patches?” query, with a very calm but mildly indignant, “Yes, yes I have.”