Read/Write Compression: Combining UnionFS and SquashFS

The third concept is called “copy up”. When a file on a read-only branch is changed, then it has to be “copied up” to some writable branch. This function allows read-only file systems, such as SquashFS, to appear so though they have read/write capabilities.

Recall that UnionFS, and really most “stackable” file systems, are just a layer of code that sits between the VFS and the underlying file system. Hence this allows any file system to be used as part of UnionFS. Consequently, you can combine ext2, ext3, ext4, xfs, reiserfs, nilfs, etc. into a single file system for users. This concept has a great deal of power and potential for combining strengths of various file systems into a coherent file system to the users.

At this time, UnionFS is still not in the mainstream kernel (it can be found in Andrew Morton’s -mm kernel). But it’s fairly easy to build and install. The next section will present coupling UnionFS and SquashFS and will discuss patching and building UnionFS.

Coupling UnionFS and SquashFS

Typically SquashFS is used for embedded devices but it shouldn’t be pigeon-holed into just those situations. Coupling it with UnionFS can provide a very good solution to a number of thorny problems: lack of space on devices and users demanding that their data remain on-line despite not accessing it or modifying it in a very long time. Using SquashFS to take data that needs to stay on-line (most of the time at a user’s request), compressing it, and then coupling it with a writable directory using UnionFS allows you to take care of these problems.

The first step is to download the UnionFS patches from the website. For this article, a CentOS 5.3 distribution was used on the following sytem:

• GigaByte MAA78GM-US2H motherboard


• An AMD Phenom II X4 920 CPUM


• 8GB of memory


• Linux 2.6.30 kernel


• The OS and boot drive are on an IBM DTLA-307020 (20GB drive at Ulta ATA/100)


• /home is on a Seagate ST1360827AS

Once you download the patch from the UnionFS website, you copy it to the root directory of the kernel source. Then you issue the following command:

patch -p1 < something.diff

At this point, there are two paths to take. The first is to enable UnionFS in the .config file in the kernel and then rebuild and reinstall the kernel. The second option is to build the UnionFS module outside the kernel. Either route is left as an exercise to the reader.

At this point UnionFS is assumed to be part of the kernel as is SquashFS and the user-space tools are installed. At this point, the test system has a user, user1, that has a directory called RESEARCH.

Figure 1 - Screen Capture of Original Directory

The directory has a mix of binary files and text files. The total directory size is,

[user1@test64 ~]$ cd RESEARCH
[user1@test64 RESEARCH]$ du -sh
890M    .

So it's a fairly large directory.

The next step is to create a SquashFS image of the directory. The snippet below shows the output of the process (notice that root performs this step).

[root@test64 squashfs-tools]# /usr/local/bin/mksquashfs /home/user1/RESEARCH /squashfs/storage/user1.RESEARCH.07042009.sqsh
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on /squashfs/storage/user1.RESEARCH.07042009.sqsh, block size 131072.
[==========================================/] 23332/23332 100%
Exportable Squashfs 4.0 filesystem, data block size 131072
        compressed data, compressed metadata, compressed fragments
        duplicates are removed
Filesystem size 237963.02 Kbytes (232.39 Mbytes)
        27.69% of uncompressed filesystem size (859274.98 Kbytes)
Inode table size 178190 bytes (174.01 Kbytes)
        28.32% of uncompressed inode table size (629191 bytes)
Directory table size 157655 bytes (153.96 Kbytes)
        44.43% of uncompressed directory table size (354862 bytes)
Number of duplicate files found 4359
Number of inodes 18890
Number of files 17698
Number of fragments 768
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1192
Number of ids (unique uids + gids) 1
Number of uids 1
        user1 (501)
Number of gids 1
        user1 (501)

Notice that the image is stored in the directory, /squashfs/storage which has been set aside for storage of SquashFS images allowing them to be stored in the same location.

This allows you to easily recognize when the image was created (assuming that you don't create images more than once a day). Also, while not required by SquashFS, the image is given the extension ".sqsh" to indicate that it is a SquashFS image.

The resulting image is much smaller than the original directory size.

[root@test64 ~]# cd /squashfs/storage/
[root@test64 storage]# ls -lsah
total 233M
4.0K drwxr-xr-x 2 root root 4.0K Jul  4 13:41 .
4.0K drwxr-xr-x 3 root root 4.0K Jul  4 13:39 ..
233M -rwx------ 1 root root 233M Jul  4 13:42 user1.RESEARCH.07042009.sqsh

The compression ratio for this image is quite large:

890M / 233M = 3.82:1

The next step is to create a directory in the user1 account called, RESEARCH.new. This is the directory where any writes to the union are stored. Also, the original directory, RESEARCH, is renamed to make sure that it is not overwritten. A very important point to make is that the directory, RESEARCH.new is owned by the user and read/writable by the user. If you decide to use a different directory you make have to chown it to the user and change the permissions using chmod.

The next step is to mount the SquashFS image. This can be done by root. In this case, this is snippet of the output of the process.

[root@test64 squashfs-tools]# mkdir /mnt/user1/
[root@test64 squashfs-tools]# mkdir /mnt/user1/RESEARCH_07042009
[root@test64 squashfs-tools]# mount -t squashfs /squashfs/storage/user1.RESEARCH.07042009.sqsh /mnt/user1/RESEARCH_07042009 -o loop
[root@test64 squashfs-tools]# ls -lsah /mnt/user1
total 12K
4.0K drwxr-xr-x  3 root  root  4.0K Jul  4 13:47 .
8.0K drwxr-xr-x  3 root  root  4.0K Jul  4 13:47 ..
   0 drwx--x--x 44 user1 user1 1.3K Jul  4 13:31 RESEARCH_07042009

Note that the mount point was created that also matches the name of the SquashFS image. Also, notice that once the image is mounted the owner is user1. Here is a screen capture of a listing of the mount point.

Figure 2 - Screen Capture of SquashFS Mount

After the SquashFS image is mounted, the UnionFS is created. The snippet below shows this process (warning - it's pretty short):

[root@test64 ~]# mount -t unionfs -o dirs=/home/user1/RESEARCH.new=rw:/mnt/user1/RESEARCH_07042009=ro unionfs /home/user1/RESEARCH
[root@test64 ~]# mount
/dev/hda3 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /home type ext3 (rw)
/dev/hda1 on /boot type ext2 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/squashfs/storage/user1.RESEARCH.07042009.sqsh on /mnt/user1/RESEARCH_07042009 type squashfs (rw,loop=/dev/loop0)
unionfs on /home/user1/RESEARCH type unionfs (rw,dirs=/home/user1/RESEARCH.new=rw:/mnt/user1/RESEARCH_07042009=ro)

There are several important things to notice with the mount command. The first directory listed in the mount command is the "top" directory that is usually the directory that is writable. In this case it is /home/user1/RESEARCH.new and is noted as "rw", that is, read/write. The second directory is the read-only SquashFS mounted image, /mnt/user1/RESEARCH_07042009 that is mounted as read-only ("ro"). Also notice that the file system "type" is unionfs. Finally, the UnionFS combination is mounted on /home/user1/RESEARCH.

A screen capture of the directory listing after the mount is shown in Figure 3 below.

Figure 3 - Screen Capture of Directory Listing Of UnionFS Mount Point

All of the files and subdirectories are there (always good to check).

The next check is to try creating a new file in the RESEARCH directory. Figure 4 below shows this process.

Figure 4 - Screen Capture of New File Creation on UnionFS Directory 'RESEARCH'

A new file called, newstuff.txt is created in the directory with a single line of text (it is a test after all). The screen capture shows the directory listing and then "cat"s the file to show that it exists.

The new file is actually created in the directory RESEARCH.new since it is the writable portion of the UnionFS combination. Figure 5 is a screen capture of the a directory listing of RESEARCH.new.

Figure 5 - Screen Capture of UnionFS Directory 'RESEARCH.new'

Notice that newly created file is listed in the RESEARCH.new directory.

If you want to make the mount command permanent, you need to modify /etc/fstab. For the example in this article, it would look like the following.

[user1@test64 RESEARCH]$ more /etc/fstab
LABEL=/                 /                       ext3    defaults        1 1
LABEL=/home             /home                   ext3    defaults        1 2
LABEL=/boot1            /boot                   ext2    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-hda2         swap                    swap    defaults        0 0
/squashfs/storage/user1.RESEARCH.07042009.sqsh /mnt/user1/RESEARCH_07042009 squashfs loop  0  0
unionfs     /home/user1/RESEARCH    unionfs    dirs=/home/user1/RESEARCH.new=rw:/mnt/user1/RESEARCH_07042009=ro  0 0

The one caveat is that you have to make sure that the SquashFS image is mounted before the unionfs combination.

It's fairly easy to use UnionFS to take a read-only SquashFS image and combine it with a local writable directory. The example shows that it's not too difficult and saved 657 MB out of 890 MB!

Summary

The combination of SquashFS and UnionFS can be a very powerful combination allowing you to potentially save lots of space via SquashFS while keeping it writable thanks to UnionFS. Assuming that the kernel supports both SquashFS and UnionFS this is an easy process.

One fun project that you can try is to use the combination of SquashFS and UnionFS to create a simple compressed personal archive for each user. Since user's are notorious for demanding that they have to have all of their data on-line at all time but yet they rarely access or the change the data, taking the "old" data, using SquashFS to create a compressed image and then mounting in the user's /home directory with a UnionFS writable section, allows you to potentially save space. It's actually not too difficult to construct.