File Testing

Intermediate ~20 min read

Robust scripts always validate files before operating on them. This lesson dives deeper into file testing patterns, teaching you to build validation functions that handle edge cases gracefully. You'll learn to write defensive code that prevents errors and provides meaningful feedback!

Comprehensive File Testing

While we covered basic file tests in the Operators module, let's explore them in the context of real validation scenarios.

#!/bin/bash
# Advanced File Testing in Bash

echo "=== File Testing Techniques ==="
echo ""

# Setup test files
test_dir="/tmp/file_test"
mkdir -p "$test_dir"
echo "test content" > "$test_dir/regular.txt"
mkdir -p "$test_dir/subdir"
ln -sf "$test_dir/regular.txt" "$test_dir/symlink.txt"
touch "$test_dir/empty.txt"

echo "Test environment:"
ls -la "$test_dir"
echo ""

# Comprehensive file tests
echo "--- Comprehensive File Tests ---"
file="$test_dir/regular.txt"
echo "Testing: $file"

[[ -e "$file" ]] && echo "  -e: EXISTS"
[[ -f "$file" ]] && echo "  -f: Is regular file"
[[ -d "$file" ]] || echo "  -d: NOT a directory"
[[ -r "$file" ]] && echo "  -r: Is readable"
[[ -w "$file" ]] && echo "  -w: Is writable"
[[ -s "$file" ]] && echo "  -s: Has content (size > 0)"
echo ""

# Testing empty file
echo "Testing empty file: $test_dir/empty.txt"
[[ -s "$test_dir/empty.txt" ]] || echo "  -s: File is EMPTY (size = 0)"
echo ""

# Testing symlink
echo "Testing symlink: $test_dir/symlink.txt"
[[ -L "$test_dir/symlink.txt" ]] && echo "  -L: Is symbolic link"
[[ -f "$test_dir/symlink.txt" ]] && echo "  -f: Points to regular file"
echo ""

# Testing directory
echo "Testing directory: $test_dir/subdir"
[[ -d "$test_dir/subdir" ]] && echo "  -d: Is directory"
[[ -x "$test_dir/subdir" ]] && echo "  -x: Is accessible (can cd into)"
echo ""

# File comparison
echo "--- File Comparison ---"
sleep 1
touch "$test_dir/newer.txt"
if [[ "$test_dir/newer.txt" -nt "$test_dir/regular.txt" ]]; then
    echo "newer.txt is newer than regular.txt"
fi

# Cleanup
rm -rf "$test_dir"

Output

Click Run to execute your code

Complete File Test Reference

Test	True When	Use Case
`-e file`	File exists (any type)	General existence check
`-f file`	Regular file exists	Before reading/writing files
`-d dir`	Directory exists	Before cd or listing
`-L file`	Symbolic link	Detecting symlinks
`-r file`	File is readable	Before cat, grep, etc.
`-w file`	File is writable	Before echo >, modification
`-x file`	File is executable	Before running scripts
`-s file`	File size > 0	Detect empty files
`-O file`	You own the file	Permission checks
`-G file`	Your group owns file	Group permission checks

Best Practice: Chain tests with && for safety: [[ -f "$file" && -r "$file" ]] && cat "$file". This ensures you only read if the file exists AND is readable.

Building Validation Functions

Encapsulate file validation logic into reusable functions for cleaner, more maintainable code.

#!/bin/bash
# File Validation Patterns

echo "=== File Validation Patterns ==="
echo ""

# Function: Validate input file
validate_file() {
    local file="$1"
    local errors=0

echo "Validating: $file"

# Check if path provided
    if [[ -z "$file" ]]; then
        echo "  ERROR: No file path provided"
        return 1
    fi

# Check existence
    if [[ ! -e "$file" ]]; then
        echo "  ERROR: File does not exist"
        return 1
    fi

# Check if regular file
    if [[ ! -f "$file" ]]; then
        echo "  ERROR: Not a regular file"
        return 1
    fi

# Check readable
    if [[ ! -r "$file" ]]; then
        echo "  ERROR: File not readable"
        ((errors++))
    fi

# Check not empty
    if [[ ! -s "$file" ]]; then
        echo "  WARNING: File is empty"
    fi

if [[ $errors -eq 0 ]]; then
        echo "  VALID: All checks passed"
        return 0
    else
        return 1
    fi
}

# Test cases
echo "--- Test Case 1: Valid file ---"
echo "test" > /tmp/valid.txt
validate_file "/tmp/valid.txt"
echo ""

echo "--- Test Case 2: Non-existent ---"
validate_file "/tmp/nonexistent_file.txt"
echo ""

echo "--- Test Case 3: Directory instead of file ---"
mkdir -p /tmp/testdir
validate_file "/tmp/testdir"
echo ""

echo "--- Test Case 4: Empty file ---"
touch /tmp/empty.txt
validate_file "/tmp/empty.txt"
echo ""

# Function: Safe file read
safe_read() {
    local file="$1"
    if [[ -f "$file" && -r "$file" ]]; then
        cat "$file"
        return 0
    else
        echo "ERROR: Cannot read file: $file" >&2
        return 1
    fi
}

echo "--- Safe Read Example ---"
safe_read "/tmp/valid.txt" && echo "(read successful)"
safe_read "/tmp/nonexistent.txt" || echo "(read failed as expected)"

# Cleanup
rm -f /tmp/valid.txt /tmp/empty.txt
rmdir /tmp/testdir 2>/dev/null

Output

Click Run to execute your code

Pro Tip: Use return codes in validation functions: return 0 for success, non-zero for failure. This lets callers use if validate_file "$f"; then ... naturally.

Safe File Operation Patterns

Combine tests into patterns that prevent errors and handle edge cases gracefully.

# Pattern 1: Safe read with fallback
safe_cat() {
    local file="$1"
    local default="$2"
    if [[ -f "$file" && -r "$file" ]]; then
        cat "$file"
    else
        echo "$default"
    fi
}
config=$(safe_cat "/etc/app.conf" "default_config")

# Pattern 2: Require file or exit
require_file() {
    local file="$1"
    if [[ ! -f "$file" ]]; then
        echo "ERROR: Required file not found: $file" >&2
        exit 1
    fi
}
require_file "/etc/critical.conf"

# Pattern 3: Wait for file with timeout
wait_for_file() {
    local file="$1"
    local timeout="${2:-30}"
    local count=0
    while [[ ! -f "$file" && $count -lt $timeout ]]; do
        sleep 1
        ((count++))
    done
    [[ -f "$file" ]]
}
wait_for_file "/tmp/ready.flag" 60 || exit 1

# Pattern 4: Process only if newer
process_if_newer() {
    local source="$1"
    local target="$2"
    if [[ "$source" -nt "$target" ]] || [[ ! -f "$target" ]]; then
        process "$source" > "$target"
    fi
}

Race Conditions: Be aware that file state can change between testing and using. For critical operations, use atomic operations like flock or write-to-temp-then-move patterns.

Common Mistakes

1. Testing without quoting

# Wrong - fails with spaces or special chars
[[ -f $filename ]]

# Correct - always quote
[[ -f "$filename" ]]

2. Assuming test order doesn't matter

# Wrong - -r test may error if file doesn't exist
[[ -r "$file" && -f "$file" ]]

# Correct - test existence first
[[ -f "$file" && -r "$file" ]]

3. Not handling empty variables

# Dangerous - if file is unset, tests current dir
[[ -d $file ]]

# Safer - use parameter expansion
[[ -d "${file:?File not specified}" ]]

# Or explicit check
[[ -n "$file" && -d "$file" ]]

Exercise: Config File Validator

Task: Create a function that validates configuration files!

Requirements:

Check file exists and is readable
Verify file is not empty
Check for required keys (e.g., "host", "port")
Return appropriate exit codes and messages

Show Solution

#!/bin/bash
# Config File Validator

validate_config() {
    local config_file="$1"
    local required_keys=("host" "port" "user")
    local errors=0

    echo "Validating: $config_file"

    # Check existence
    if [[ ! -f "$config_file" ]]; then
        echo "  ERROR: File not found"
        return 1
    fi

    # Check readable
    if [[ ! -r "$config_file" ]]; then
        echo "  ERROR: File not readable"
        return 1
    fi

    # Check not empty
    if [[ ! -s "$config_file" ]]; then
        echo "  ERROR: File is empty"
        return 1
    fi

    # Check required keys
    for key in "${required_keys[@]}"; do
        if ! grep -q "^$key=" "$config_file"; then
            echo "  ERROR: Missing required key: $key"
            ((errors++))
        fi
    done

    if [[ $errors -eq 0 ]]; then
        echo "  SUCCESS: Config is valid"
        return 0
    else
        echo "  FAILED: $errors error(s) found"
        return 1
    fi
}

# Test
config="/tmp/test.conf"
cat > "$config" << 'EOF'
host=localhost
port=8080
user=admin
EOF

validate_config "$config"
rm -f "$config"

Summary

Always Test: Check file existence and permissions before operations
Test Order: Check existence (-f) before permissions (-r, -w)
Quote Variables: Always quote paths in tests: [[ -f "$file" ]]
Use Functions: Encapsulate validation logic for reusability
Return Codes: Use 0 for success, non-zero for failure
Error Messages: Write to stderr: echo "error" >&2

What's Next?

Now let's explore Working with Directories. You'll learn to traverse directories, process files recursively, and use the find command for powerful directory operations!

Previous File Manipulation

Next Working with Directories