Size Can Matter: Throughput Performance with a Disk-Based Journal – Part 4
Turning from Metadata performance to throughput performance, we examines the impact of journal size on ext4 when the journal is disk-based. Dig into the numbers and see what you can do to improve throughput performance.
Wednesday, February 3rd, 2010
Summary
The biggest surprise is the lack of change in performance as the journal size is changed. It was anticipated that there would be some changes, particularly for the small block case because there are so many files. However, there is little change in the throughput as the journal size is changed. But, as with previous throughput tests, there can be a very large change in performance when the block size is changed.
You may be disappointed in reading this article and finding there is little change in the throughput a the journal size is increased (when the journal is on a separate disk). But remember that sometimes a negative answer is just as good as a positive answer. Also remember that the results are limited to the parameter variations in this article. Perhaps for a 1KB record size, there could be some large changes in throughput with journal size. But increasing the number of parameter variations just increases the number of tests resulting in longer to run the full range of tests.
Test Results
As mentioned previously, there are 13 tests that are each run 10 times and there are 4 record sizes. This makes a total of 520 tests that were run per file system.
Because of the large number of tests that are run, the results are split into two groups. The first group is for the write tests: write, re-write, random write, record re-write, fwrite, refwrite. The second group is for the read tests: read, re-read, random reads, backwards read, strided read, fread, and refread. Each table below is for one of the two groups for a specific record size (1MB, 4MB, 8MB, 16MB). So that means there are 8 tables of results.
The first two tables of results are for the 1MB record size. Table 1 below presents the throughput in KB/s for the file systems for the 6 write tests for all four journal sizes and both journal devices.
Table 1 – IOzone Write Performance Results with a Record Length of 1MB, a File Size of 16GB, Disk based Journal, Four Journal Sizes
| Journal Size and Device |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk – 16MB |
111,657.8
357.98 |
111,043.6
143.87 |
67,165.3
387.89 |
2,968,590.2
16,307.28 |
111,344.9
144.30 |
110,899.5
137.29 |
| Disk – 64MB |
111,530.5
770.87 |
110,916.0
108.50 |
67,120.6
487.88 |
2,969,452.3
54,934.38 |
111,403.7
219.65 |
110,974.8
96.27 |
| Disk – 256MB |
111,606.8
440.43 |
110,997.0
109.20 |
67,363.6
307.09 |
2,976,033.7
33,476.51 |
111,435.7
137.92 |
111,025.8
78.28 |
| Disk – 1GB |
111,583.9
352.17 |
110.970.8
116.91 |
67,046.0
298.97 |
2,982,673.1
41,942.17 |
111,469.2
139.51 |
108,638.4
6,900.21 |
Table 2 below presents the throughput in KB/s for the file systems for the 7 read tests for a record length of 1MB.
Table 2 – IOzone read Performance Results with a Record Length of 1MB, a File Size of 16GB, Disk based Journal, Four Journal Sizes
| Journal Size and Device |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk – 16MB |
101,234.8
49.81 |
101,297.1
49.16 |
54,487.9
36.23 |
75,928.2
89.76 |
51,562.3
57.04 |
101,321.6
18.82 |
101,315.0
49.88 |
| Disk – 64MB |
101,225.5
39.29 |
101,298.3
26.67 |
54,465.1
46.25 |
75,850.0
175.41 |
51,602.5
32.40 |
101,304.8
38.86 |
101,310.1
34.34 |
| Disk – 256MB |
101,233.4
44.08 |
101,309.1
30.62 |
54,413.5
27.24 |
75,933.8
90.04 |
51,534.1
30.00 |
101,309.6
47.09 |
101,329.2
38.05 |
| Disk – 1GB |
101,239.9
7.74 |
101,283.5
52.54 |
54,389.4
38.81 |
75,915.8
46.72 |
51,568.8
21.87 |
101,308.4
37.16 |
101,319.0
49.25 |
The next two tables of results are for the 4MB record size. Table 3 below presents the throughput in KB/s for the file systems for the 6 write tests.
Table 3 – IOzone Write Performance Results with a Record Length of 4MB, a File Size of 16GB, Disk based Journal, Four Journal Sizes
| Journal Size and Device |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk – 16MB |
111,685.5
75.01 |
110,924.6
72.57 |
76,813.0
703.51 |
2,454,743.8
204,498.2 |
111,381.9
153.85 |
110,932.9
102.27 |
| Disk – 64MB |
111,654.7
113.79 |
110,908.4
52.05 |
77,045.3
687.05 |
2,506,107.5
103,023.81 |
111,386.1
157.88 |
108,701.8
6,709.12 |
| Disk – 256MB |
111,659.1
96.49 |
110,919.3
65.95 |
77,066.4
747.78 |
2,483,751.8
1104,088.49 |
111,396.1
118.94 |
110,922.1
93.49 |
| Disk – 1GB |
111,614.8
60.95 |
110,930.4
43.33 |
77,332.6
622.51 |
2,535,905.1
58,923.02 |
111,358.8
191.65 |
110,935.3
86.49 |
Table 4 below presents the throughput in KB/s for the file systems for the 7 read tests for a record length of 4MB.
Table 4 – IOzone Read Performance Results with a Record Length of 4MB and a File Size of 16GB
| File System |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk – 16MB |
101,216.2
57.74 |
101,296.5
37.66 |
89,324.0
126.72 |
105,153.9
114.73 |
91,017.8
100.20 |
101,304.9
24.8 |
101,314.0
51.7 |
| Disk – 64MB |
101,225.3
29.53 |
101,290.2
55.39 |
89,277.5
69.32 |
105,182.0
98.03/font> |
90,979.3
71.40 |
101,259.5
29.12 |
101,317.4
27.67 |
| Disk – 256MB |
101,233.5
38.92 |
101,295.5
34.54 |
89,302.0
55.66 |
105,142.6
97.67 |
90,979.9
74.28 |
101,273.7
40.52 |
101,323.1
40.90 |
| Disk – 1GB |
101,219.3
54.56 |
101.296.9
32.19 |
89,309.7
186.99 |
105,075.9
83.94 |
90,998.5
74.87 |
101,286.4
24.02 |
101,312.4
49.67 |
The next two tables of results are for the 8MB record size. Table 5 below presents the throughput in KB/s for the file systems for the 6 write tests.
Table 5 – IOzone Write Performance Results with a Record Length of 8MB and a File Size of 16GB
| File System |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk – 16MB |
111,639.1
100.59 |
110,912.1
87.79 |
79,392.2
885.54 |
1,529,967.9
33,394.28 |
111,247.6
284.43 |
110,945.5
92.18 |
| Disk – 64MB |
111,646.5
77.51 |
110,999.0
64.95 |
79,922.5
677.89 |
1,526,391.3
38,053.05 |
111,365.6
111.48 |
110,900.9
82.39 |
| Disk – 256MB |
111,683.5
75.24 |
110,948.0
102.33 |
79,689.1
853.39 |
1,529,227.2
27,714.88 |
111,280.4
186.3 |
110,959.4
87.35 |
| Disk – 1GB |
111,671.1
50.23 |
110,952.2
74.86 |
79,004.5
1,036.98 |
1,545,638.1
16,741.76 |
111,280.8
163.38 |
110,950.84
86.4 |
Table 6 below presents the throughput in KB/s for the file systems for the 7 read tests for a record length of 8MB.
Table 6 – IOzone Read Performance Results with a Record Length of 8MB and a File Size of 16GB
| File System |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk – 16MB |
101,282.6
42.55 |
101,316.6
41.41 |
100,510.4
53.56 |
112,082.8
423.22 |
103,605.7
83.85 |
101,258.4
39.66 |
101,316.5
19.42 |
| Disk – 64MB |
101,287.4
60.46 |
101,305.2
38.78 |
100,477.6
50.99 |
112,253.1
143.28 |
103,567.0
111.48 |
101,245.4
39.40 |
101,301.2
18.93 |
| Disk – 256MB |
101,304.2
48.55 |
101,325.5
51.92 |
100,478.8
53.14 |
112,168.0
115.61 |
103,608.3
125.20 |
101,262.4
26.93 |
101,295.6
40.10 |
| Disk – 1GB |
101,286.4
61.52 |
101,314.4
52.10 |
100,479.6
46.89 |
112,235.7
109.66 |
103,620.0
163.38 |
101,262.6
40.22 |
101,298.1
49.63 |
The final two tables of results are for the 16MB record size. Table 7 below presents the throughput in KB/s for the file systems for the 6 write tests.
Table 7 – IOzone Write Performance Results with a Record Length of 16MB and a File Size of 16GB
| File System |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk – 16MB |
111,605.6
130.38 |
110,899.3
92.21 |
83,469.7
739.94 |
1,509,892.0
14,934.62 |
111,453.2
162.66 |
111,056.4
104.33 |
| Disk – 64MB |
111,647.7
92.44 |
110,919.4
74.03 |
83,454.5
824.25 |
1,499,743.7
33,576.16 |
111,553.3
77.26 |
111,041.5
75.58 |
| Disk – 256MB |
111,652.4
131.09 |
110,932.5
69.59 |
83,042.1
828.34 |
1,491,108.7
17,523.31 |
111,421.5
139.31 |
111,077.6
66.49 |
| Disk – 1GB |
111,692.3
74.56 |
110,915.7
82.9 |
83,209.5
897.76 |
1,489,433.3
18,731.89 |
111,594.8
117.81 |
106,347.9
9,239.07 |
Table 9 below presents the throughput in KB/s for the file systems for the 7 read tests for a record length of 16MB.
Table 8 – IOzone Read Performance Results with a Record Length of 16MB and a File Size of 16GB
| File System |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk – 16MB |
101,231.7
15.82 |
101,281.2
35.64 |
105,773.5
45.32 |
116,283.0
95.72 |
110,337.8
50.20 |
101,211.7
38.22 |
101,280.4
36.30 |
| Disk – 64MB |
101,231.3
37.30 |
101,307.6
26.35 |
105,904.4
56.13 |
116,253.7
84.08 |
110,337.8
50.20 |
101,213.8
55.26 |
101,272.1
39.37 |
| Disk – 256MB |
101,226.9
37.05 |
101,293.4
48.51 |
105,904.1
34.79 |
116,175.5
287.92 |
110,375.4
101.42 |
101,217.0
49.72 |
101,282.5
30.50 |
| Disk – 1GB |
101,215.4
46.2 |
101,313.0
21.13 |
105,879.2
49.70 |
116,279.4
114.21 |
110,330.4
75.65 |
101,210.9
35.62 |
101,292.1
27.17 |
Jeff Layton is an Enterprise Technologist for HPC at Dell. He can be found lounging around at a nearby Frys enjoying the coffee and waiting for sales (but never during working hours).
|
|
