EROFS Deep Dive: Enhanced Read-Only File System

Introduction

EROFS (Enhanced Read-Only File System) is a lightweight, high-performance read-only filesystem in the Linux kernel. Originally developed by Gao Xiang at Huawei, it was merged in Linux 4.19 (October 2018) and has since become the default read-only filesystem for Android system partitions (Android 13+), container images, and embedded Linux systems.

This guide provides an in-depth technical exploration of EROFS internals, on-disk format, compression algorithms, performance characteristics, and real-world deployment on Linux systems.

Design Philosophy

EROFS is built around three core principles:

1. Fixed-size output compression — decompression cost is O(1) per page, not O(compression-block-size). This eliminates the read amplification problem that plagues SquashFS.

2. Minimal metadata overhead — no journaling, no complex B-tree indices, just simple flat structures optimized for read-only access.

3. Security-first — immutable by design, bit-for-bit identical across deployments, ideal for verified boot chains.

graph LR
    subgraph "Traditional Read-Write FS"
        EXT4["ext4<br>journal + bitmap + tree<br>Complex, mutable"]
    end
    subgraph "SquashFS"
        SQ["SquashFS<br>compression blocks<br>Read amplification on random I/O"]
    end
    subgraph "EROFS"
        EROFS["EROFS<br>fixed-size output<br>O(1) random access"]
    end
    EXT4 -->|"Too complex<br>for read-only"| SQ
    SQ -->|"Read amplification<br>problem"| EROFS
    style EROFS fill:#38a169,color:#fff
    style SQ fill:#d69e2e,color:#000
    style EXT4 fill:#3182ce,color:#fff

On-Disk Format

Superblock

The EROFS superblock is located at byte offset 1024 (block 1 on 1024-byte blocks):

/* From fs/erofs/erofs_fs.h */
#define EROFS_SUPER_MAGIC_V1    0xE0F5E1E2
#define EROFS_ISLOTBITS         5
#define EROFS_SLOTSIZE          (1 << EROFS_ISLOTBITS)  /* 32 bytes */

struct erofs_super_block {
    __le32 magic;           /* 0xE0F5E1E2 */
    __le32 checksum;        /* crc32c of superblock */
    __le32 feature_compat;  /* compatible features */
    __le32 blkszbits;       /* block size = 1 << blkszbits */
    __le8  sb_extslots;     /* superblock extended slots */
    __le8  root_nid;        /* root inode nid */
    __le16 inos;            /* inode count (not used) */
    __le32 build_time;      /* filesystem build time */
    __le32 build_time_nsec;
    __le32 blocks;          /* total blocks */
    __le32 meta_blkaddr;    /* metadata block address */
    __le8  uuid[16];        /* filesystem UUID */
    __le16 volume_name[16]; /* volume label (UTF-16) */
    __le32 feature_incompat;/* incompatible features */
    __le8  compr_level;     /* compression level (v2+) */
    __le8  uuid2[16];       /* extended UUID (v2+) */
    /* ... additional fields ... */
} __attribute__((packed));

Inode Structure

EROFS uses compact 32-byte inodes with two variants:

graph TB
    subgraph "Compact Inode (32 bytes)"
        CI["i_format (2B) | i_xattr (2B) | i_mode (2B)"]
        CI2["i_nlink (2B) | i_size (4B)"]
        CI3["i_ino (4B) | i_uid (2B) | i_gid (2B)"]
        CI4["i_mtime (4B) | i_mtime_nsec (4B)"]
        CI5["i_inline_data (6B) | i_reserved (2B)"]
    end
    subgraph "Extended Inode (64 bytes)"
        EI["Same header + extended fields"]
        EI2["i_ctime (4B) | i_ctime_nsec (4B)"]
        EI3["i_crtime (4B) | i_crtime_nsec (4B)"]
    end
    CI --> EI

Inode data layout modes:

Compact inode data layout:
┌──────────────────────────────────────────────┐
│ Mode               │ Description             │
├──────────────────────────────────────────────┤
│ FLAT_PLAIN         │ Block list (extents)     │
│ FLAT_INLINE        │ Inline data + extents    │
│ FLAT_COMPRESSION   │ Compressed clusters      │
│ CHUNK_BASED        │ Fixed-size chunks (v5.15)│
└──────────────────────────────────────────────┘

Directory Entries

EROFS directories use a hash-based lookup table for fast name resolution:

struct erofs_dirent {
    __le16 nid;         /* inode number */
    __le8  nameoff;     /* offset to name in block */
    __le8  file_type;   /* DT_REG, DT_DIR, etc. */
    /* followed by name data */
};

Lookup uses a compact index:

Directory block layout:
┌────────────────────────────────────┐
│ Header: nchildren, reserved        │
│ Index[0]: hash → name offset       │
│ Index[1]: hash → name offset       │
│ ...                                │
│ Name data: "file1\0file2\0..."     │
└────────────────────────────────────┘

Compression Architecture

Fixed-Size Output Compression (FOC)

This is EROFS’s key innovation. Traditional compression (SquashFS) uses fixed-size input blocks, which means reading a single 4KB page requires decompressing an entire compression block (often 128KB-1MB):

SquashFS (fixed input):
┌─────────────────────────────────────┐
│ Compression block (256KB input)     │
│ → Compressed to variable size      │
│ → Read 4KB: decompress 256KB       │
│ → 64x read amplification!          │
└─────────────────────────────────────┘

EROFS (fixed output):
┌─────────────────────────────────────┐
│ Cluster (e.g., 4KB output × 32)    │
│ → Each 4KB page decompresses alone │
│ → Read 4KB: decompress ~4KB        │
│ → 1x read amplification            │
└─────────────────────────────────────┘

graph LR
    subgraph "SquashFS Decompression"
        SQ_READ["Read 4KB page"] --> SQ_DECOMP["Decompress 256KB block"]
        SQ_DECOMP --> SQ_EXTRACT["Extract 4KB"]
    end
    subgraph "EROFS Decompression"
        E_READ["Read 4KB page"] --> E_DECOMP["Decompress 4KB cluster"]
        E_DECOMP --> E_DONE["Done"]
    end
    style SQ_DECOMP fill:#e53e3e,color:#fff
    style E_DECOMP fill:#38a169,color:#fff

Compression Algorithms

EROFS supports two compression backends:

LZ4 (Default)

# Create EROFS with LZ4 compression
mkfs.erofs -zlz4 /dev/loop0 /source/dir
mkfs.erofs -zlz4hc /dev/loop0 /source/dir  # High compression

LZ4 characteristics:

• Speed: ~7800 MB/s decompression
• Ratio: 2.1x average on typical Linux rootfs
• CPU cost: Negligible
• Use case: Default for most deployments

LZMA

# Create EROFS with LZMA compression
mkfs.erofs -zlzma /dev/loop0 /source/dir

LZMA characteristics:

• Speed: ~80 MB/s decompression
• Ratio: 3.5x average
• CPU cost: High
• Use case: Extreme space constraints (embedded, live CDs)

DEFLATE (Linux 6.4+)

# Create EROFS with DEFLATE (libdeflate)
mkfs.erofs -zdeflate /dev/loop0 /source/dir

Comparison

Compression benchmark (Android system partition, ~2.1GB):
┌──────────────┬──────────┬──────────┬────────────┬──────────────┐
│ Algorithm    │ Size     │ Ratio    │ Compress   │ Decompress   │
├──────────────┼──────────┼──────────┼────────────┼──────────────┤
│ None         │ 2100 MB  │ 1.0x     │ -          │ 12000 MB/s   │
│ LZ4          │ 1000 MB  │ 2.1x     │ 500 MB/s   │ 7800 MB/s    │
│ LZ4HC        │ 920 MB   │ 2.3x     │ 40 MB/s    │ 7800 MB/s    │
│ LZMA         │ 600 MB   │ 3.5x     │ 8 MB/s     │ 80 MB/s      │
│ DEFLATE      │ 850 MB   │ 2.5x     │ 100 MB/s   │ 1200 MB/s    │
└──────────────┴──────────┴──────────┴────────────┴──────────────┘

graph LR
    subgraph "Compression Ratio vs Speed"
        direction TB
        LZ4["LZ4: 2.1x ratio<br>7800 MB/s decompress"]
        LZ4HC["LZ4HC: 2.3x ratio<br>7800 MB/s decompress"]
        DEFLATE["DEFLATE: 2.5x ratio<br>1200 MB/s decompress"]
        LZMA["LZMA: 3.5x ratio<br>80 MB/s decompress"]
    end
    LZ4 -->|"Better ratio"| LZ4HC
    LZ4HC -->|"Better ratio"| DEFLATE
    DEFLATE -->|"Best ratio"| LZMA
    style LZ4 fill:#38a169,color:#fff
    style LZMA fill:#e53e3e,color:#fff

Chunk-Based Compression (Linux 5.15+)

For large files, EROFS supports chunk-based compression where files are split into fixed-size chunks, each compressed independently:

# Create with chunk size
mkfs.erofs -C 65536 -zlz4 /dev/loop0 /source/dir

Chunk-based layout:
┌─────────────────────────────────────────┐
│ Chunk 0 (64KB) → compressed → stored   │
│ Chunk 1 (64KB) → compressed → stored   │
│ Chunk 2 (64KB) → compressed → stored   │
│ ...                                     │
│ Each chunk decompresses independently   │
│ Random access within chunk = O(1)       │
└─────────────────────────────────────────┘

Kernel Configuration

Building EROFS Support

# Check current support
zgrep EROFS /proc/config.gz
# or
grep EROFS /boot/config-$(uname -r)

# Required kernel config options:
CONFIG_EROFS_FS=m                  # EROFS filesystem module
CONFIG_EROFS_FS_XATTR=y            # Extended attributes
CONFIG_EROFS_FS_POSIX_ACL=y        # POSIX ACLs
CONFIG_EROFS_FS_SECURITY=y         # SELinux/AppArmor labels
CONFIG_EROFS_FS_ZIP=y              # LZ4 compression
CONFIG_EROFS_FS_ZIP_LZMA=y         # LZMA compression
CONFIG_EROFS_FS_ZIP_DEFLATE=y      # DEFLATE compression (6.4+)
CONFIG_EROFS_FS_ONDEMAND=y         # On-demand loading (FUSE-based)
CONFIG_EROFS_FS_PCPU_KTHREAD=y     # Per-CPU decompression threads

Loading and Mounting

# Load module
modprobe erofs

# Basic mount
mount -t erofs /dev/sdb1 /mnt/erofs

# Mount compressed image
mount -t erofs -o ro /path/to/image.erofs /mnt/erofs

# Verify filesystem
dmesg | grep erofs
mount | grep erofs

Creating EROFS Images

mkfs.erofs Usage

# Install erofs-utils
sudo apt install erofs-utils      # Debian/Ubuntu
sudo dnf install erofs-utils      # Fedora
sudo pacman -S erofs-utils        # Arch

# Basic image creation
mkfs.erofs image.erofs /source/directory

# With LZ4 compression
mkfs.erofs -zlz4 image.erofs /source/directory

# With LZMA (high compression)
mkfs.erofs -zlzma image.erofs /source/directory

# Custom block size
mkfs.erofs -b 4096 -zlz4 image.erofs /source/directory

# With chunk-based compression
mkfs.erofs -C 65536 -zlz4 image.erofs /source/directory

# With UUID
mkfs.erofs -U 12345678-1234-1234-1234-123456789abc image.erofs /source/directory

# Verbose output
mkfs.erofs -zlz4 --all-root image.erofs /source/directory

Multi-Threaded Compression

# Use multiple threads for compression (faster build)
mkfs.erofs -T $(nproc) -zlz4 image.erofs /source/directory

# Benchmark: building a 2GB rootfs
# 1 thread:  45s
# 4 threads: 14s
# 8 threads: 8s

Building Container Images

# Create EROFS container rootfs from Docker image
CONTAINER=$(docker create alpine:latest)
docker export $CONTAINER | mkfs.erofs -zlz4 alpine.erofs

# With Podman
podman create alpine:latest
podman export $CONTAINER | mkfs.erofs -zlz4 alpine.erofs

# Mount and verify
mkdir -p /tmp/alpine
mount -t erofs alpine.erofs /tmp/alpine
ls /tmp/alpine

Performance Characteristics

Random Read Performance

Random 4KB reads from compressed 2GB image (NVMe SSD):
┌────────────────────┬──────────┬──────────┬──────────┐
│ Filesystem         │ IOPS     │ Latency  │ CPU      │
├────────────────────┼──────────┼──────────┼──────────┤
│ ext4 (uncompressed)│ 85,000   │ 12μs     │ 2%       │
│ EROFS (LZ4)        │ 72,000   │ 14μs     │ 5%       │
│ EROFS (LZMA)       │ 3,200    │ 312μs    │ 45%      │
│ SquashFS (LZ4)     │ 18,000   │ 55μs     │ 12%      │
│ SquashFS (LZMA)    │ 1,800    │ 556μs    │ 52%      │
│ Btrfs (LZO, cow)   │ 70,000   │ 14μs     │ 4%       │
└────────────────────┴──────────┴──────────┴──────────┘

graph LR
    subgraph "Random 4KB Read IOPS"
        direction TB
        EXT4["ext4: 85,000<br>████████████████████████"]
        EROFS_LZ4["EROFS LZ4: 72,000<br>██████████████████"]
        SQ_LZ4["SquashFS LZ4: 18,000<br>████"]
        EROFS_LZMA["EROFS LZMA: 3,200<br█"]
        SQ_LZMA["SquashFS LZMA: 1,800<br>▏"]
    end
    style EXT4 fill:#3182ce,color:#fff
    style EROFS_LZ4 fill:#38a169,color:#fff
    style SQ_LZ4 fill:#d69e2e,color:#000

Sequential Read Performance

Sequential read throughput (1MB blocks, NVMe SSD):
┌────────────────────┬──────────┬──────────┬──────────┐
│ Filesystem         │ Throughput│ CPU      │ Ratio    │
├────────────────────┼──────────┼──────────┼──────────┤
│ ext4               │ 3400 MB/s│ 3%       │ 1.0x     │
│ EROFS (LZ4)        │ 3200 MB/s│ 8%       │ 0.94x    │
│ EROFS (LZMA)       │ 80 MB/s  │ 95%      │ 0.02x    │
│ SquashFS (LZ4)     │ 2800 MB/s│ 10%      │ 0.82x    │
│ Btrfs (LZO)        │ 3100 MB/s│ 5%       │ 0.91x    │
└────────────────────┴──────────┴──────────┴──────────┘

Mount Time

Time to mount a 2GB rootfs image:
┌────────────────────┬──────────┬──────────┐
│ Filesystem         │ Mount    │ Total    │
├────────────────────┼──────────┼──────────┤
│ ext4               │ 50ms     │ 50ms     │
│ EROFS (LZ4)        │ 15ms     │ 15ms     │
│ EROFS (LZMA)       │ 15ms     │ 15ms     │
│ SquashFS (LZ4)     │ 80ms     │ 80ms     │
│ SquashFS (LZMA)    │ 120ms    │ 120ms    │
└────────────────────┴──────────┴──────────┘

EROFS has near-instant mount time because it only reads the superblock and root inode. The rest is loaded on demand.

Memory Usage

In-kernel memory overhead (2GB image, 100k files):
┌────────────────────┬──────────┬──────────┐
│ Filesystem         │ Kernel   │ Per-file │
│                    │ Memory   │ Memory   │
├────────────────────┼──────────┼──────────┤
│ ext4               │ ~45MB    │ ~450B    │
│ EROFS              │ ~8MB     │ ~80B     │
│ SquashFS           │ ~12MB    │ ~120B    │
└────────────────────┴──────────┴──────────┘

Android Integration

Why Android Chose EROFS

Google selected EROFS for Android 13+ system partitions because:

1. Faster app launches — random reads from compressed system libraries
2. Smaller OTA updates — better compression than ext4
3. Lower RAM usage — minimal kernel memory for mounted filesystem
4. Security — immutable system partition, verified boot

Android Build Integration

# In Android build system (AOSP)
BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := erofs
BOARD_SYSTEMIMAGE_COMPRESSOR := lz4hc

# Vendor partition
BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE := erofs
BOARD_VENDORIMAGE_COMPRESSOR := lz4

# Product partition
BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE := erofs

Performance on Android

App cold start time (Android 13, Pixel 7):
┌────────────────────┬──────────┬──────────┐
│ Partition FS       │ Chrome   │ Maps     │
├────────────────────┼──────────┼──────────┤
│ ext4               │ 1.2s     │ 0.9s     │
│ EROFS (LZ4)        │ 0.8s     │ 0.6s     │
│ EROFS (LZ4HC)      │ 0.75s    │ 0.55s    │
│ SquashFS (LZ4)     │ 1.1s     │ 0.85s    │
└────────────────────┴──────────┴──────────┘

Container Image Optimization

EROFS vs OverlayFS

For container storage, EROFS can be combined with OverlayFS for an immutable base layer with a writable top layer:

# Mount EROFS as lower layer with writable overlay
mount -t erofs container-base.erofs /mnt/lower
mount -t overlay overlay \
    -o lowerdir=/mnt/lower,upperdir=/var/lib/overlay/upper,workdir=/var/lib/overlay/work \
    /mnt/container

graph TB
    subgraph "Container Layers"
        UPPER["Upper (writable)<br>ext4 / tmpfs"]
        LOWER["Lower (EROFS)<br>Compressed, read-only"]
        MERGED["Merged View<br>OverlayFS"]
    end
    MERGED --> UPPER
    MERGED --> LOWER
    style UPPER fill:#e53e3e,color:#fff
    style LOWER fill:#38a169,color:#fff
    style MERGED fill:#3182ce,color:#fff

Image Size Comparison

Container image size (Alpine 3.18):
┌────────────────────┬──────────┬──────────┐
│ Format             │ Size     │ Savings  │
├────────────────────┼──────────┼──────────┤
│ OCI tar+gzip       │ 3.3 MB   │ -        │
│ ext4 (uncompressed)│ 8.2 MB   │ -148%    │
│ EROFS (LZ4)        │ 3.8 MB   │ -15%     │
│ EROFS (LZ4HC)      │ 3.4 MB   │ -3%      │
│ EROFS (LZMA)       │ 2.8 MB   │ +15%     │
│ SquashFS (LZ4)     │ 4.1 MB   │ -24%     │
└────────────────────┴──────────┴──────────┘

Advanced Features

Extended Attributes (xattr)

# Set xattr before creating image
setfattr -n security.selinux -v "u:object_r:system_file:s0" /source/file

# Create image preserving xattrs
mkfs.erofs -zlz4 --xattr-include="*" image.erofs /source/dir

# Mount with xattr support
mount -t erofs -o xattr image.erofs /mnt
getfattr -n security.selinux /mnt/file

On-Demand Loading (EROFS on FUSE)

Linux 5.16+ supports mounting EROFS images from remote sources using a FUSE-based on-demand loader:

# Build with on-demand support
CONFIG_EROFS_FS_ONDEMAND=y

# Mount with on-demand URL
mount -t erofs -o ondemand_loop=/dev/loop0 image.erofs /mnt

This enables:

• Lazy loading of container images from registries
• Partial image mounting for embedded systems
• Network-mounted filesystem images

FS Verity Integration

# Enable fs-verity on EROFS files
fsverity enable /mnt/erofs/important-file

# Verify integrity
fsverity measure /mnt/erofs/important-file

Direct I/O

# Mount with direct I/O (bypass page cache)
mount -t erofs -o dax image.erofs /mnt

Debugging and Inspection

erofs-utils Tools

# Dump filesystem info
dump.erofs image.erofs

# Extract files
extract.erofs -x image.erofs -o /output/dir

# List contents
erofs.fsck image.erofs --dry-run

# Verify integrity
erofs.fsck image.erofs

Kernel Debugging

# Enable EROFS debug messages
echo 'module erofs +p' > /sys/kernel/debug/dynamic_debug/control

# Check EROFS stats
cat /proc/fs/erofs/sb_info

# Trace EROFS operations
trace-cmd record -p block -e erofs
trace-cmd report | grep erofs

perf Analysis

# Profile EROFS read operations
perf record -g -e block:block_rq_issue -- mount -t erofs image.erofs /mnt
perf report

# Trace decompression latency
perf probe --add 'erofs_readpages'
perf record -e probe:erofs_readpages -aR sleep 10

Use Cases

Embedded Linux

# Minimal EROFS rootfs for embedded (ARM64)
mkfs.erofs -C 65536 -zlzma --all-root \
    -b 4096 embedded-rootfs.erofs /build/rootfs

# Size: ~50MB compressed vs ~200MB uncompressed
# RAM: ~2MB kernel memory for mount

Live CDs/USBs

# Create bootable EROFS live image
mkfs.erofs -zlz4 live-rootfs.erofs /live/rootfs

# In GRUB config
linux /vmlinuz root=/dev/sr0 rootfstype=erofs
initrd /initrd.img

Immutable Server Images

# Build immutable server image
mkfs.erofs -zlz4hc --uuid $(uuidgen) \
    server-image.erofs /chroot

# Deploy: mount read-only, overlay writable /var and /etc
mount -t erofs server-image.erofs /
mount -t tmpfs tmpfs /var
mount -t tmpfs tmpfs /etc

Comparison with Other Read-Only Filesystems

Summary

EROFS represents a significant advancement in read-only filesystem design for Linux. Its fixed-size output compression model eliminates the read amplification problem that has limited SquashFS for decades, while maintaining excellent compression ratios.

Key takeaways:

1. Android’s default — EROFS is the system partition filesystem for Android 13+
2. O(1) random access — no read amplification like SquashFS
3. Minimal footprint — small kernel module, low memory usage
4. Fast mount — near-instant mount time (superblock + root inode only)
5. Container-friendly — excellent as OverlayFS lower layer
6. Security-first — immutable, verifiable, fs-verity compatible

For any read-only use case on Linux — container images, embedded systems, live media, or immutable server deployments — EROFS should be the first choice over SquashFS.