Filesystem metadata overhead


Filesystem metadata overhead

Which filesystems have the largest metadata overhead[1]? We can find out using guestfish sparse file support and a short shell script:

#!/bin/sh –

guestfish=fish/guestfish
testimg=/mnt/tmp/test/test.img
size=1G

for fstype in ext2 ext3 ext4 xfs ntfs msdos ; do
$guestfish <
sparse $testimg $size
run
sfdiskM /dev/sda ,
mkfs $fstype /dev/sda1
EOF
echo -n “$fstype: ”
du $testimg
done

The results:
Filesystem Allocated kilobytes (out of 1G) Overhead %
ext2 16948 1.6%
ext3 [2] 33352 3.2%
ext4 [2] 33288 3.2%
xfs 5132 0.5%
ntfs [3] 5748 0.5%
msdos & vfat 2076 0.2%
reiserfs [3] 32916 3.1%
btrfs [3] 4224 0.4%
hfs & hfsplus [3] 16432 1.6%
nilfs2 [3] 2060 0.2%
jfs [3] 4364 0.4%
gfs [3] 16612 1.6%
gfs2 [3,4] 132576 12%
Notes

[1] Yes, there are shortcomings in the methodology. What this really measures is how many blocks are written by the mkfs program. Even if you believe the figures, this only measures the initial overhead, but you can use the same technique to measure the overhead of storing (eg) lots of small files, or whatever is appropriate for your workload.
[2] Note the difference between ext2 and ext3/4 seems to be entirely down to the size of the journal, which is a kind of metadata overhead, but one that you can easily control.
[3] Needs three small patches to libguestfs to get these filesystems to work.
[4] This seems off the scale — needs further investigation.

Get article from

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s