Sometimes we'll need a newer copy of the userspace tools for our file system of choice, so let's find out how to build these tools. Also provide some usage examples.

btrfs

btrfs-progs

Build btrfs-progs:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git btrfs-progs-git
cd btrfs-progs-git

sudo apt-get install e2fslibs-dev libacl1-dev libattr1-dev libblkid-dev liblzo2-dev uuid-dev     # Debian or Ubuntu
sudo dnf install e2fsprogs-devel libacl-devel libattr-devel libblkid-devel lzo-devel uuid-devel  # Fedora
make && sudo make prefix=/opt/btrfs-progs/ install

Note: we might still need a patch for this to compile:

Usage

TBD

BTRFS and Free Space - Emergency Response (2019-02-11)
Fixing Btrfs Filesystem Full Problems (2014-05-04)

Ext4

e2fsprogs

The e2fsprogs can be obtained via Git:

git clone git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git e2fsprogs-git
cd e2fsprogs-git

./configure --prefix=/opt/e2fsprogs --enable-htree
make && sudo make install

Usage

We can use tune2fs to modify some file system properties. Let's have the device checked every 20 mounts or 6 months, and turn it read-only in case of errors:

tune2fs -c 20 -i 6m -e remount-ro $DEV

e2image

We can use e2image to create an image of the file system:

e2image -rap $DEV disk.img                  # -r  create a raw image
                                            # -a  include all data
                                            # -p  show progress while creating the image

Restore the image with:

e2image -rap disk.img $DEV

dump

To clone a file system on file system level (as opposed to file or block level), we can use dump.

Now let's dump the currently mounted ext4 file system of to another disk, $DEV:

$ mount -t ext4 $DEV /mnt/disk/
$ cd /mnt/disk/

$ dump -0 -f - / | restore -rf -
  DUMP: Date of this level 0 dump: Thu Aug 24 23:30:19 2017
  DUMP: Dumping /dev/sda1 (/) to standard output
  DUMP: Label: none
  DUMP: Writing 10 Kilobyte records
  DUMP: mapping (Pass I) [regular files]
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 2708627 blocks.
  DUMP: Volume 1 started with block 1 at: Thu Aug 24 23:30:19 2017
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
restore: ./lost+found: File exists
 ./tmp/rstdir1503642619: (inode 805) not found on tape
 ./tmp/rstmode1503642619: (inode 951) not found on tape
  DUMP: Volume 1 completed at: Thu Aug 24 23:31:02 2017
  DUMP: Volume 1 2701050 blocks (2637.74MB)
  DUMP: Volume 1 took 0:00:43
  DUMP: Volume 1 transfer rate: 62815 kB/s
  DUMP: 2701050 blocks (2637.74MB)
  DUMP: finished in 43 seconds, throughput 62815 kBytes/sec
  DUMP: Date of this level 0 dump: Thu Aug 24 23:30:19 2017
  DUMP: Date this dump completed:  Thu Aug 24 23:31:02 2017
  DUMP: Average transfer rate: 62815 kB/s
  DUMP: DUMP IS DONE

Since this was our root disk and we want to boot from the clone, we need to install the bootloader as well:

mount -t devtmpfs udev /mnt/disk/dev/
mount -t proc proc /mnt/disk/proc/
mount -t sysfs sysfs /mnt/disk/sys/
chroot /mnt/disk/

> grub-install --recheck /dev/sdc
Installing for i386-pc platform.
Installation finished. No error reported.

> update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.9.0-0.bpo.2-amd64
Found initrd image: /boot/initrd.img-4.9.0-0.bpo.2-amd64
 No volume groups found
 Found Debian GNU/Linux (8.9) on /dev/sdc
 done

As a final step, check the bootloader's configuration and /etc/fstab if the device names reflect the new disk, $DEV.

JFS

jfsutils

JFS for Linux:

cvs -z3 -d:pserver:anonymous@jfs.cvs.sourceforge.net:/cvsroot/jfs co -P jfsutils
mv jfsutils{,-cvs}
cd jfsutils-cvs

sudo apt-get install uuid-dev                                             # Debian or Ubuntu
sudo dnf install uuid-devel                                               # Fedora
./autogen.sh --prefix=/opt/jfsutils
make && sudo make install

Usage

TBD

OverlayFS

Usage

Basic OverlayFS usage - in this case, we want to make use of a read-only NFS share:

mkdir -p /mnt/nfs /usr/local/src /mnt/overlay_{upper,work}
mount -t nfs -o ro server:/data /mnt/nfs

mount -t overlay -o lowerdir=/mnt/nfs,upperdir=/mnt/overlay_upper,workdir=/mnt/overlay_work \
     overlayfs /usr/local/src

With that, we can still write to /usr/local/src, although the read-only NFS share would not support this:

$ touch /usr/local/src/foo
$ ls -l /mnt/nfs /usr/local/src
/mnt/nfs:
drwx--xr-x 1 dummy staff    50 Sep  2  2015 dir_name
-rw-r--r-- 1 dummy staff   892 Feb 27  2015 file.exe

/usr/local/src:
drwx--xr-x 1 dummy staff    50 Sep  2  2015 dir_name
-rw-r--r-- 1 dummy staff   892 Feb 27  2015 file.exe
-rw------- 1 dummy staff     0 Aug 14 23:32 foo

ReiserFS

reiserfsprogs

Build reiserfsprogs:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeffm/reiserfsprogs.git reiserfsprogs-git
cd reiserfsprogs-git

sudo apt-get install libacl1-dev                                          # Debian or Ubuntu
sudo dnf install libacl-devel                                             # Fedora
libtoolize --copy --install --force && aclocal && autoheader && autoconf && automake --add-missing
./configure --prefix=/opt/reiserfsprogs
make && make install

Usage

TBD

XFS

xfsprogs

Build xfsprogs:

git clone https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git xfsprogs-git
cd xfsprogs-git

sudo apt-get install automake gettext libattr1-dev libblkid-dev libicu-dev libtool m4 pkg-config uuid-dev         # Debian or Ubuntu
sudo dnf     install automake gettext libattr-devel libblkid-devel libtool m4 pkgconf-pkg-config libuuid-devel    # Fedora
make configure && ./configure --prefix=/opt/xfsprogs
make && sudo make install
sudo chown -R root:root /opt/xfsprogs && sudo chmod -R a+rX /opt/xfsprogs

`xfs_scrub`

We might want to^[1] try another xfsprogs branch supporting xfs_scrub^[2], along with kernel support for the same feature.

cd /usr/local/src/linux-git/
git remote add djwong-xfs-linux https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git
git fetch --tags djwong-xfs-linux
git checkout -b local-xfs djwong-xfs-linux/master

Build and install the kernel. While this is going on, we can install the userspace part too:

cd /usr/local/src/xfsprogs-git
git remote add djwong-xfsprogs-dev https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git
git fetch --tags djwong-xfsprogs-dev 
git checkout -b local-djwong djwong-xfsprogs-dev/djwong-devel
make configure && ./configure --prefix=/opt/xfsprogs-dev && make -j8 && sudo make install

xfstests

The xfstests is not exactly a filesystem checking tool, but a full blown file system test suite.

Install prerequisites:

For Debian or Ubuntu:

sudo apt-get install acl attr autoconf automake bc btrfs-progs dbench dump e2fsprogs fio gawk gcc git jfsutils libacl1-dev libaio-dev libattr1-dev libcap2-bin libgdbm-dev libssl-dev libtool-bin lvm2 quota reiser4progs reiserfsprogs uuid-dev xfsdump xfslibs-dev xfsprogs

Fedora:

sudo dnf install acl attr autoconf automake bc btrfs-progs dbench dump e2fsprogs fio gawk gcc gdbm-devel git jfsutils libacl-devel libaio-devel libattr-devel libcap libtool libuuid-devel lvm2 openssl-devel quota reiserfs-utils uuid-devel xfsdump xfsprogs xfsprogs-devel

openSUSE:

sudo zypper install acl attr autoconf automake btrfsprogs dbench dump e2fsprogs fio gawk gcc gdbm-devel git-core jfsutils libacl-devel libaio-devel libattr-devel libcap-progs libopenssl-devel libtool libuuid-devel lvm2 make quota reiserfs uuid-devel xfsdump xfsprogs xfsprogs-devel

Checkout the source:

git clone https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git xfstests-git
cd xfstests-git

make configure
./configure --prefix=/opt/xfstests && make
sudo make install                                            # Optional

id fsgqa || sudo useradd -m -U fsgqa                         # Add the fsgqa test user & group and create
                                                             # its home directory

Set up some environment variables:

export TEST_DEV=/dev/mapper/vg0-lv0 TEST_DIR=/mnt/test SCRATCH_DEV=/dev/mapper/vg0-lv1 SCRATCH_MNT=/mnt/scratch

mkdir -p ${TEST_DIR} ${SCRATCH_MNT}
mkfs.xfs ${TEST_DEV}

Run (a single) test:

cd /opt/xfstests/xfstests/
./check tests/xfs/002

Follow along with:

tail -f /opt/xfstests/xfstests/results/xfs/002.full

Usage

Create a filesystem with metadata checksums and a separate free inode btree index:

mkfs.xfs -m crc=1,finobt=1 /dev/sdX

Check for fragmentation:^[3]^[4]

$ ulimit -d unlimited                                                          # Otherwise xfs_db may fail^[5]
$ xfs_db -c frag -r /dev/sde1
actual 922480, ideal 28640, fragmentation factor 96.90%

Online de-fragment with xfs_fsr, the XFS filesystem reorganizer:

xfs_fsr -v /mnt/disk

Check (and repair) an XFS filesystem:

xfs_repair /dev/sde1

If errors cannot be recovered at once, one might want to save a xfs_metadump copy before continuing, so that the issue can be analyzed later on.

`xfs_quota`

TBD

References

[1] Re: xfs_repair: couldn't map inode 2089979520, err = 117

[2] xfs_scrub: create online filesystem scrub program

[3] Use xfs_fsr to keep your XFS filesystem optimal

[4] XFS: Defragmentation

[5] Re: XFS repair problem

[1]

[2]

[3]

[4]

[5]

File Systems

Contents

btrfs

btrfs-progs

Usage

Ext4

e2fsprogs

Usage

e2image

dump

JFS

jfsutils

Usage

OverlayFS

Usage

ReiserFS

reiserfsprogs

Usage

XFS

xfsprogs

`xfs_scrub`

xfstests

Usage

`xfs_quota`

References

Navigation menu

Search