componefs is a new file system proposed for Linux
Late last year we share here on the blog information about a new file system you were working on Alexander Larsson, the creator of Flatpak and now on this occasion he has announced the launch of the first stable version file system ComposeFS.
ComposeFS is a “new” file system that is optimized to efficiently share the contents of multiple mounted disk images. In practice, ComposeFS can be useful for mounting container images and hosting a Git-like OSTree repository.
About ComposeFS
At the beginning of ComposeFS development, Its implementation was intended as a separate module of the Linux kernel, but as time goes by In development work, Alexander Larsson mentions that He realized that promoting a new FS in the main kernel structure It's much more complicated About what I thought. It is because of that the developers changed tactics and reworked the project in the form of a plugin about FS OverlayFS and EROFS, which are already present in the kernel, whose functionality partially overlaps with ComposeFS. Therefore, work to integrate ComposeFS support into the Linux kernel has been reduced to promoting patches to OverlayFS and EROFS that implement the specific capabilities required for FS.
It is mentioned that EROFS functionality meets ComposeFS requirements to starting with kernel version 5.15. The changes to OverlayFS were made in several stages: In kernel 6.5, support was included for “data-only” layers, which are used only for data (separated from metadata).
The remaining changes, along with the ability to store fs-verity hashes in the extended overlay.verity (xattr) attribute, were recently adopted in the 6.6-rc1 test kernel, marking the inclusion of all functionality needed to run Composefs on Linux.
Alexander Larsson mentions that thanks to the adoption of all the necessary changes in the kernel, it was possible to determine and correct the final image format of ComposeFS and with which the first stable version of the file system could be released, stabilizing the storage format and the API/ABI library.
Main new features in ComposeFS 1.0
Among the functional differences of this stable version with respect to previous developments, the introduction of optimizations that increase the efficiency of the storage format, In addition, the use of tools built into the fs-verity kernel subsystem to verify digital signatures is now avoided in favor of libraries that work in user space.
Another highlight is the composefs-info utility to inspect image files of ComposeFS and identify inconsistencies between metadata and separately stored data.
In addition, to mount ComposeFS, the FUSE module loaded in userspace composefs-fuse and the mount.composefs utility are now used, and the mkcomposefs utility is provided to create the FS.
For check the content of individual files and the full image under shared storage conditions, fs-verity mechanism is used which, when accessing files, checks the correspondence of the hashes specified in the binary index with the actual content, if an attacker makes a change to a file in the base directory or the data is corrupted as a result of a failure, such reconciliation will reveal a discrepancy.
It is worth mentioning that of the projects that already use ComposeFS, there are Ostree and Container Storage Library. Ostree's git repository fully supports ComposeFS, including content verification capabilities, but the code remains marked as experimental for now.
The container storage library provides an initial implementation of a backend that uses ComposeFS to store container images. Once completed, the backend can be used to use ComposeFS for deduplication and anti-spoofing of images managed using Podman.
Finally if you are interested in knowing more about it, you can check the details in the following link.