Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

xargs: Argument Processing

Introduction

xargs builds and executes command lines from standard input. It bridges the gap between commands that produce output (like find, grep -l) and commands that need arguments. xargs is essential for handling large argument lists, parallel execution, and safe filename processing.

Basic Usage

# Simple: pass stdin as arguments
echo "file1 file2 file3" | xargs rm

# Equivalent to:
rm file1 file2 file3

# From a file
cat filenames.txt | xargs wc -l

# Limit arguments per invocation
echo "1 2 3 4 5 6" | xargs -n2 echo
# Output:
# 1 2
# 3 4
# 5 6

Delimiter Handling

Default Delimiter

By default, xargs splits on spaces, tabs, and newlines:

# Default: splits on whitespace
echo "file1 file2 file3" | xargs echo
# Output: file1 file2 file3

# Multiple lines
printf "file1\nfile2\nfile3\n" | xargs echo
# Output: file1 file2 file3

Null Delimiter (-0)

The safest way to handle filenames with spaces, quotes, or special characters:

# Null-separated (from find -print0)
find . -name "*.txt" -print0 | xargs -0 grep "pattern"

# Null-separated input from printf
printf "file 1.txt\0file 2.txt\0" | xargs -0 ls -la

# WRONG: breaks on filenames with spaces
find . -name "*.txt" | xargs grep "pattern"

Custom Delimiter (-d)

# Use colon as delimiter
echo "one:two:three" | xargs -d: echo
# Output: one two three

# Use newline as delimiter
printf "file1\nfile2\nfile3" | xargs -d'\n' echo
# Output: file1 file2 file3

# Use comma as delimiter
echo "a,b,c,d" | xargs -d, -n2 echo
# Output:
# a b
# c d

Placeholder (-I{})

Replace {} with each input item:

# Basic placeholder
echo "file1 file2 file3" | xargs -I{} cp {} /backup/

# Custom placeholder
echo "file1 file2 file3" | xargs -I% cp % /backup/

# With find
find . -name "*.py" -print0 | xargs -0 -I{} cp {} /backup/

# Multiple lines output
find . -name "*.log" -print0 | xargs -0 -I{} sh -c '
    echo "=== {} ==="
    tail -5 "{}"
'

# Move and rename
find . -name "*.jpeg" -print0 | xargs -0 -I{} sh -c '
    mv "{}" "${1%.jpeg}.jpg"
' _

# Placeholder with complex commands
find . -type f -name "*.bak" -print0 | xargs -0 -I{} sh -c '
    size=$(stat -c%s "{}")
    echo "{}: $size bytes"
'

-I{} vs -i

# -I{} is standard (POSIX)
echo "file1" | xargs -I{} echo {}

# -i{} is deprecated but still works
echo "file1" | xargs -i{} echo {}

# -I{} replaces ALL occurrences
echo "file1" | xargs -I{} echo {} and {}
# Output: file1 and file1

Parallel Execution (-P)

Run multiple commands simultaneously:

# Run 4 jobs in parallel
find . -name "*.py" -print0 | xargs -0 -P4 python3 -m py_compile

# Parallel compression
find . -name "*.log" -print0 | xargs -0 -P8 gzip

# Parallel image conversion
find . -name "*.png" -print0 | xargs -0 -P4 -I{} convert {} {}.jpg

# Parallel with progress
find . -name "*.txt" -print0 | xargs -0 -P8 -I{} sh -c '
    wc -l "{}"
' | awk '{total+=$1; count++} END {print count, "files,", total, "lines"}'

# Use all CPU cores
nproc=$(nproc)
find . -name "*.c" -print0 | xargs -0 -P"$nproc" gcc -c

-P Behavior

# -P0: run as many jobs as possible (unlimited)
find . -name "*.py" -print0 | xargs -0 -P0 pylint

# -P4: exactly 4 parallel jobs
find . -name "*.py" -print0 | xargs -0 -P4 pylint

# Combined with -n (batch size)
find . -name "*.txt" -print0 | xargs -0 -P4 -n10 wc -l
# Runs 4 parallel wc processes, each with 10 files

Argument Limits

Max Arguments (-n)

# Process 5 files at a time
find . -name "*.txt" -print0 | xargs -0 -n5 wc -l

# One file at a time (like -I{} but without placeholder)
find . -name "*.txt" -print0 | xargs -0 -n1 wc -l

# Two arguments per invocation
echo "a b c d e f" | xargs -n2 echo
# Output:
# a b
# c d
# e f

Max Command Length (-s)

# Limit command line length
find . -name "*.txt" -print0 | xargs -0 -s 4096 echo

# Default is usually 131072 bytes
# Useful when command has other arguments
find . -name "*.txt" -print0 | xargs -0 -s 4096 grep -i "pattern" /dev/null {}

Handling Argument Overflow

# When there are too many arguments, xargs runs the command multiple times
seq 1 100000 | xargs echo    # Runs multiple echo invocations

# Show what would happen
seq 1 100000 | xargs -t echo

# Dry run: show commands without executing
seq 1 100 | xargs -t -n10 echo

xargs with find

Safe Patterns

# CORRECT: null-separated
find . -name "*.txt" -print0 | xargs -0 grep "pattern"

# CORRECT: -exec + (similar, no pipe)
find . -name "*.txt" -exec grep "pattern" {} +

# WRONG: breaks on special filenames
find . -name "*.txt" | xargs grep "pattern"

# Safe pattern for all cases
find . -name "*.txt" -print0 | xargs -0 -I{} sh -c '
    grep "pattern" "$1"
' _

Complex Pipelines

# Find, filter, and process
find . -name "*.log" -mtime +7 -print0 | \
    xargs -0 -P4 gzip

# Find with multiple actions
find . -name "*.py" -print0 | xargs -0 sh -c '
    for f in "$@"; do
        lines=$(wc -l < "$f")
        if [ "$lines" -gt 1000 ]; then
            echo "$f: $lines lines"
        fi
    done
' _

# Find and backup
find . -name "*.conf" -print0 | \
    xargs -0 tar -czf config_backup_$(date +%Y%m%d).tar.gz

Safety Features

Interactive Mode (-p)

# Prompt before each execution
find . -name "*.bak" -print0 | xargs -0 -p rm
# rm file1.bak file2.bak ?...y

# With -I{} for per-file prompting
find . -name "*.bak" -print0 | xargs -0 -p -I{} rm {}
# rm file1.bak ?...y
# rm file2.bak ?...n

Verbose Mode (-t)

# Print command before executing
find . -name "*.txt" -print0 | xargs -0 -t grep "pattern"
# grep pattern file1.txt file2.txt file3.txt

No Run If Empty (-r)

# Don't run if stdin is empty (GNU extension)
find . -name "*.nonexistent" -print0 | xargs -0 -r rm
# Without -r: rm (no arguments) → error
# With -r: nothing happens

# Note: GNU xargs doesn't run with empty input by default
# but POSIX requires -r for this behavior

Exit Status

# xargs exit codes:
# 0 - All invocations succeeded
# 123 - One or more invocations failed
# 124 - Command exited with status 255
# 125 - Command killed by signal
# 126 - Command cannot execute
# 127 - Command not found
# 1-120 - Exit status from last command

# Check exit status
find . -name "*.py" -print0 | xargs -0 python3 -m py_compile
if [ $? -eq 0 ]; then
    echo "All files compiled successfully"
fi

# Use with set -e
set -e
find . -name "*.py" -print0 | xargs -0 python3 -m py_compile

Practical Examples

File Processing

# Count lines in all Python files
find . -name "*.py" -print0 | xargs -0 wc -l | tail -1

# Find and remove empty files
find . -type f -empty -print0 | xargs -0 rm

# Batch rename
find . -name "*.jpeg" -print0 | xargs -0 -I{} sh -c '
    mv "$1" "${1%.jpeg}.jpg"
' _

# Archive old files
find /var/log -name "*.log.*" -mtime +30 -print0 | \
    xargs -0 tar -czf /backup/old_logs_$(date +%Y%m%d).tar.gz

# Convert encoding
find . -name "*.txt" -print0 | xargs -0 -I{} sh -c '
    iconv -f ISO-8859-1 -t UTF-8 "$1" > "$1.utf8" && mv "$1.utf8" "$1"
' _

Code Analysis

# Find TODO/FIXME comments
find . -name "*.py" -not -path "*/venv/*" -print0 | \
    xargs -0 grep -Hn "TODO\|FIXME\|HACK"

# Run linter on changed files
git diff --name-only --diff-filter=ACM -- '*.py' | \
    xargs -d'\n' -I{} pylint {}

# Count function definitions
find . -name "*.py" -print0 | \
    xargs -0 grep -c "^def " | \
    awk -F: '{sum+=$2} END {print sum, "functions"}'

# Find files with no newline at end
find . -type f -name "*.py" -print0 | xargs -0 -I{} sh -c '
    [ "$(tail -c 1 "$1" | wc -l)" -eq 0 ] && echo "$1"
' _

System Administration

# Check disk usage of large files
find / -type f -size +100M -print0 2>/dev/null | \
    xargs -0 ls -lh 2>/dev/null | sort -k5 -rh | head -20

# Update file permissions
find /var/www -type f -print0 | xargs -0 chmod 644
find /var/www -type d -print0 | xargs -0 chmod 755

# Kill processes by pattern
ps aux | grep "[z]ombie" | awk '{print $2}' | xargs kill -9

# Backup configuration files
find /etc -name "*.conf" -mtime -7 -print0 | \
    xargs -0 tar -czf /backup/etc_changes_$(date +%Y%m%d).tar.gz

Network Operations

# Check URLs from file
cat urls.txt | xargs -P10 -I{} curl -s -o /dev/null -w "%{http_code} {}\n" {}

# Download files in parallel
cat urls.txt | xargs -P4 -I{} wget -q {}

# Ping sweep
seq 1 254 | xargs -P254 -I{} ping -c1 -W1 192.168.1.{} | grep "bytes from"

Advanced Patterns

Combining with Other Tools

# xargs + awk
find . -name "*.log" -print0 | xargs -0 awk '{print NR, $0}'

# xargs + sed
find . -name "*.py" -print0 | xargs -0 sed -i 's/old/new/g'

# xargs + grep + cut
find . -name "*.py" -print0 | \
    xargs -0 grep -h "^import " | \
    sort | uniq -c | sort -rn

# xargs + parallel (GNU parallel alternative)
find . -name "*.py" -print0 | xargs -0 -P8 -n1 python3 -m py_compile

Conditional Execution

# Only run if find produces output
files=$(find . -name "*.tmp" -print0)
[ -n "$files" ] && echo "$files" | xargs -0 rm

# With xargs -r (GNU)
find . -name "*.tmp" -print0 | xargs -0 -r rm

# Check each file before processing
find . -name "*.bak" -print0 | xargs -0 -I{} sh -c '
    if [ -f "{}" ] && [ -w "{}" ]; then
        rm "{}"
    else
        echo "Cannot remove: {}"
    fi
'

Error Handling

# Continue on errors
find . -name "*.py" -print0 | xargs -0 python3 -m py_compile || true

# Log errors
find . -name "*.py" -print0 | xargs -0 -I{} sh -c '
    if ! python3 -m py_compile "$1" 2>/dev/null; then
        echo "Compilation failed: $1" >&2
    fi
' _

# With timeout
find . -name "*.py" -print0 | xargs -0 -I{} timeout 30 python3 {}

xargs vs Alternatives

xargs vs -exec

# xargs: builds single command line (more efficient)
find . -name "*.txt" -print0 | xargs -0 grep "pattern"

# find -exec: runs command per file (or per batch with +)
find . -name "*.txt" -exec grep "pattern" {} +

# xargs advantages:
# - Parallel execution (-P)
# - More control over argument grouping (-n, -s)
# - Can pipe to other commands

# find -exec advantages:
# - No pipe needed
# - Works on systems without xargs
# - Handles special characters naturally with +

xargs vs GNU Parallel

# xargs: simple parallel execution
find . -name "*.py" -print0 | xargs -0 -P4 python3 -m py_compile

# GNU parallel: more features
find . -name "*.py" -print0 | parallel -j4 python3 -m py_compile

# GNU parallel advantages:
# - Progress bar
# - Job logging
# - Remote execution
# - Better argument handling
# - ETA and job control

# Install GNU parallel
sudo apt install parallel

xargs vs while read

# xargs: efficient, handles many files
find . -name "*.txt" -print0 | xargs -0 grep "pattern"

# while read: more flexible, per-file logic
find . -name "*.txt" -print0 | while IFS= read -r -d '' file; do
    if grep -q "pattern" "$file"; then
        echo "Found in: $file"
    fi
done

# Use xargs for simple commands
# Use while read for complex per-file logic

Common Pitfalls

1. Spaces in Filenames

# WRONG
find . -name "*.txt" | xargs rm    # Breaks on "my file.txt"

# CORRECT
find . -name "*.txt" -print0 | xargs -0 rm

2. Empty Input

# GNU xargs: safe by default (doesn't run with empty input)
find . -name "*.nonexistent" | xargs echo  # Nothing happens

# POSIX xargs: may run with no arguments
# Use -r flag (GNU) or -0 (always safe)
find . -name "*.nonexistent" -print0 | xargs -0 -r rm

3. Command Line Too Long

# xargs automatically splits when command line is too long
seq 1 1000000 | xargs echo    # Runs multiple echo invocations

# Check with -t to see what happens
seq 1 1000000 | xargs -t -n100 echo

4. Quoting Issues

# xargs handles quoting automatically
echo "'hello world'" | xargs echo    # hello world

# But be careful with shell expansion
echo "*.txt" | xargs echo    # *.txt (literal, not expanded)
echo "*.txt" | xargs sh -c 'echo $1' _    # *.txt (still literal)
echo "*.txt" | xargs bash -c 'echo $1' _  # *.txt
# Use eval if you need expansion (dangerous)
echo "*.txt" | xargs -I{} sh -c 'eval echo {}'  # DANGEROUS

xargs Environment and Signals

Environment Variables

# xargs inherits the current environment
export MY_VAR="hello"
echo "file.txt" | xargs -I{} sh -c 'echo $MY_VAR {}'

# xargs does NOT pass stdin to the executed command
# stdin is used for the argument list
echo "input" | xargs cat file.txt    # cat reads file.txt, not stdin

# Use -a to read from file instead of stdin
xargs -a filenames.txt echo

Signal Handling

# xargs forwards signals to child processes
# SIGTERM, SIGINT are propagated

# With parallel jobs, all children receive the signal
find . -name "*.py" -print0 | xargs -0 -P4 python3 -m py_compile
# Ctrl+C kills all parallel processes

# Timeout individual commands
find . -name "*.py" -print0 | xargs -0 -I{} timeout 30 python3 {}

Quoting and Escaping

# xargs handles quoting from stdin automatically
printf "'hello world'\n" | xargs echo
# Output: hello world (quotes stripped)

# Single quotes, double quotes, and backslashes are handled
printf '"hello world"\n' | xargs echo
# Output: hello world

# Backslash escaping
printf 'hello\ world\n' | xargs echo
# Output: hello world

# Show quoting with -p
printf "hello world\n" | xargs -p echo
# echo hello world ?...

GNU xargs vs BSD xargs (macOS)

FeatureGNU xargsBSD xargs
-0 (null delimiter)✅✅
-r (no run if empty)✅ (default)❌ (needs flag)
-I{} placeholder✅✅
-P parallel✅✅ (macOS 10.13+)
-d custom delimiter✅❌
-a read from file✅❌
Multiple commands✅Limited
 
# Portable: always use -r on macOS
find . -name "*.tmp" -print0 | xargs -0r rm

# macOS install GNU xargs
brew install findutils    # Installs as gxargs

References