Working with Directories

Intermediate ~25 min read

Directories are the backbone of file organization. This lesson teaches you to navigate, iterate, and process directory contents efficiently. You'll master multiple approaches from simple globs to the powerful find command for handling complex directory operations!

Basic Directory Operations

Start with fundamental directory operations: listing contents, iteration, and counting items.

#!/bin/bash
# Working with Directories in Bash

echo "=== Directory Operations ==="
echo ""

# Setup test directory structure
base="/tmp/dir_test"
mkdir -p "$base"/{docs,scripts,logs}
touch "$base/docs/readme.txt"
touch "$base/docs/guide.md"
touch "$base/scripts/backup.sh"
touch "$base/scripts/deploy.sh"
touch "$base/logs/app.log"
touch "$base/config.txt"

echo "Test directory structure:"
find "$base" -type f
echo ""

# Basic directory operations
echo "--- Basic Directory Operations ---"
echo "Current directory: $(pwd)"
echo "Home directory: $HOME"
echo ""

# List directory contents
echo "--- Listing Contents ---"
echo "Files in $base:"
ls -la "$base"
echo ""

# Iterate over files in directory
echo "--- Iterating with Glob ---"
for file in "$base"/*; do
    if [[ -f "$file" ]]; then
        echo "File: $(basename "$file")"
    elif [[ -d "$file" ]]; then
        echo "Dir:  $(basename "$file")/"
    fi
done
echo ""

# Count items
echo "--- Counting ---"
file_count=$(find "$base" -type f | wc -l)
dir_count=$(find "$base" -type d | wc -l)
echo "Files: $file_count"
echo "Directories: $dir_count"

# Cleanup
rm -rf "$base"

Output

Click Run to execute your code

Key Directory Commands

Command	Purpose	Common Options
`ls`	List directory contents	`-l` long, `-a` all, `-h` human
`cd`	Change directory	`-` previous, `~` home
`pwd`	Print working directory	`-P` physical path
`mkdir`	Create directories	`-p` parents, `-m` mode
`rmdir`	Remove empty directories	`-p` parents

Brace Expansion: Create multiple directories at once: mkdir -p project/{src,bin,docs,tests}. This creates all four subdirectories in a single command.

Recursive Directory Processing

Processing files in subdirectories requires recursive techniques. Here are four different approaches.

#!/bin/bash
# Recursive Directory Processing

echo "=== Recursive Directory Processing ==="
echo ""

# Setup test structure
base="/tmp/recursive_test"
mkdir -p "$base/level1/level2/level3"
touch "$base/root.txt"
touch "$base/level1/file1.txt"
touch "$base/level1/level2/file2.txt"
touch "$base/level1/level2/level3/file3.txt"

echo "Test structure:"
find "$base" -type f
echo ""

# Method 1: find with -exec
echo "--- Method 1: find -exec ---"
echo "Processing all .txt files:"
find "$base" -name "*.txt" -exec echo "Found: {}" \;
echo ""

# Method 2: find with while loop
echo "--- Method 2: find with while ---"
while IFS= read -r -d '' file; do
    echo "Processing: $file"
done < <(find "$base" -name "*.txt" -print0)
echo ""

# Method 3: Recursive function
echo "--- Method 3: Recursive Function ---"
process_dir() {
    local dir="$1"
    local indent="${2:-}"

for item in "$dir"/*; do
        [[ -e "$item" ]] || continue
        if [[ -d "$item" ]]; then
            echo "${indent}[DIR] $(basename "$item")"
            process_dir "$item" "  $indent"
        else
            echo "${indent}[FILE] $(basename "$item")"
        fi
    done
}
process_dir "$base"
echo ""

# Method 4: globstar (Bash 4+)
echo "--- Method 4: globstar (**) ---"
shopt -s globstar
for file in "$base"/**/*.txt; do
    [[ -f "$file" ]] && echo "Globstar found: $file"
done
shopt -u globstar

# Cleanup
rm -rf "$base"

Output

Click Run to execute your code

Choosing the Right Method

Method	Pros	Cons
`find -exec`	Handles special characters, powerful filters	Spawns process per file
`find \| while`	Complex processing per file	Subshell variable scope
Recursive function	Full control, custom logic	More code, stack limits
`globstar **`	Simple syntax, readable	Bash 4+ only, less flexible

Performance Tip: For many files, use find ... -exec command {} + (plus instead of semicolon) to batch files into fewer command executions.

Advanced find Operations

The find command is incredibly powerful. Master these patterns for complex file operations.

#!/bin/bash
# Advanced find Operations

echo "=== Advanced find for Directories ==="
echo ""

# Setup test environment
base="/tmp/find_adv"
mkdir -p "$base"/{src,build,docs}
touch "$base/src/main.c"
touch "$base/src/utils.c"
touch "$base/build/main.o"
touch "$base/build/app"
touch "$base/docs/readme.md"

# Create some varied files
echo "old content" > "$base/old.txt"
touch -d "30 days ago" "$base/old.txt"
dd if=/dev/zero of="$base/large.bin" bs=1K count=500 2>/dev/null

echo "Test structure created"
echo ""

# Find and delete empty directories
echo "--- Find Empty Directories ---"
mkdir -p "$base/empty_dir"
find "$base" -type d -empty -print
echo ""

# Find by multiple extensions
echo "--- Find Multiple Extensions ---"
find "$base" $ -name "*.c" -o -name "*.md" $ -type f
echo ""

# Find and execute with multiple commands
echo "--- Find with Actions ---"
echo "Files with sizes:"
find "$base" -type f -exec ls -lh {} \; | awk '{print $9, $5}'
echo ""

# Find excluding directories
echo "--- Find Excluding Paths ---"
echo "All files except in build/:"
find "$base" -path "$base/build" -prune -o -type f -print
echo ""

# Find by size ranges
echo "--- Find by Size ---"
echo "Files between 100KB and 1MB:"
find "$base" -type f -size +100k -size -1M
echo ""

# Find modified files
echo "--- Find by Time ---"
echo "Files modified more than 7 days ago:"
find "$base" -type f -mtime +7
echo ""

# Find and transform
echo "--- Find and Process ---"
echo "Source files with line counts:"
find "$base/src" -name "*.c" -exec wc -l {} \;

# Cleanup
rm -rf "$base"

Output

Click Run to execute your code

Essential find Expressions

# By type
find . -type f           # Regular files only
find . -type d           # Directories only
find . -type l           # Symbolic links

# By name (supports globs)
find . -name "*.log"     # Case-sensitive match
find . -iname "*.LOG"    # Case-insensitive

# By size
find . -size +100M       # Larger than 100MB
find . -size -1k         # Smaller than 1KB
find . -empty            # Zero-size files/dirs

# By time (days)
find . -mtime -7         # Modified in last 7 days
find . -atime +30        # Accessed over 30 days ago
find . -newer ref.txt    # Newer than ref.txt

# By permissions
find . -perm 755         # Exact permissions
find . -perm -u+x        # User executable

# Combining expressions
find . -type f -name "*.sh" -executable
find . \( -name "*.c" -o -name "*.h" \)
find . -type f ! -name "*.bak"

find Actions

# Print (default)
find . -name "*.txt" -print

# Execute command for each
find . -name "*.sh" -exec chmod +x {} \;

# Execute with confirmation
find . -name "*.bak" -ok rm {} \;

# Print with format
find . -type f -printf "%p %s bytes\n"

# Delete (careful!)
find . -name "*.tmp" -delete

# Batch execution (more efficient)
find . -name "*.c" -exec cat {} +

Caution: Always test find commands with -print before using -delete or destructive -exec actions. The deletion is immediate and permanent!

Practical Directory Patterns

# Find and delete empty directories
find . -type d -empty -delete

# Find large files
find . -type f -size +100M -exec ls -lh {} \;

# Find recently modified files
find . -type f -mmin -60  # Modified in last 60 minutes

# Find and replace in files
find . -name "*.txt" -exec sed -i 's/old/new/g' {} \;

# Find files not matching pattern
find . -type f ! -name "*.md" ! -name "*.txt"

# Count files by extension
find . -type f -name "*.js" | wc -l

# Find duplicate files by size
find . -type f -exec ls -l {} \; | awk '{print $5}' | sort | uniq -d

# Safe recursive delete with confirmation
find_delete() {
    find "$1" -name "$2" -print
    read -p "Delete these files? [y/N] " confirm
    [[ "$confirm" == [yY] ]] && find "$1" -name "$2" -delete
}

# Process files in parallel (GNU parallel)
# find . -name "*.jpg" | parallel convert {} -resize 50% resized/{}

Common Mistakes

1. Forgetting to quote glob patterns

# Wrong - shell expands glob before find runs
find . -name *.txt

# Correct - find receives the pattern
find . -name "*.txt"

2. Using semicolon without backslash

# Wrong - shell interprets semicolon
find . -exec echo {} ;

# Correct - escape or quote
find . -exec echo {} \;
find . -exec echo {} ';'

3. Iterating with ls output

# Wrong - breaks on spaces/special chars
for f in $(ls); do echo "$f"; done

# Correct - use glob directly
for f in *; do echo "$f"; done

# Or use find with null separator
find . -print0 | while IFS= read -r -d '' f; do
    echo "$f"
done

Exercise: Directory Statistics

Task: Create a script that generates directory statistics!

Requirements:

Count total files and directories
Calculate total size
Find largest file
Count files by extension
Show most recently modified files

Show Solution

#!/bin/bash
# Directory Statistics Tool

dir_stats() {
    local dir="${1:-.}"

    echo "=== Directory Statistics: $dir ==="
    echo ""

    # Count files and directories
    local files=$(find "$dir" -type f | wc -l)
    local dirs=$(find "$dir" -type d | wc -l)
    echo "Files: $files"
    echo "Directories: $dirs"
    echo ""

    # Total size
    local size=$(du -sh "$dir" 2>/dev/null | cut -f1)
    echo "Total size: $size"
    echo ""

    # Largest files
    echo "Top 5 largest files:"
    find "$dir" -type f -exec ls -lh {} \; 2>/dev/null | \
        sort -k5 -hr | head -5 | \
        awk '{print "  " $5 " " $9}'
    echo ""

    # Files by extension
    echo "Files by extension:"
    find "$dir" -type f -name "*.*" | \
        sed 's/.*\.//' | sort | uniq -c | \
        sort -rn | head -10 | \
        awk '{print "  " $1 " ." $2}'
    echo ""

    # Recently modified
    echo "Recently modified (last 24h):"
    find "$dir" -type f -mtime -1 -exec ls -lh {} \; 2>/dev/null | \
        head -5 | awk '{print "  " $6 " " $7 " " $8 " " $9}'
}

# Run on current directory or specified path
dir_stats "${1:-.}"

Summary

Basic Ops: Use ls, cd, pwd, mkdir -p for fundamentals
Iteration: Use globs (for f in *) for simple cases
Recursive: Use find for recursive processing with filters
globstar: Enable with shopt -s globstar for ** patterns
find Actions: Use -exec {} \; for per-file, {} + for batched
Safety: Test destructive commands with -print first

What's Next?

Congratulations on completing the File Operations module! You now have powerful skills for reading, writing, testing, and navigating files and directories. Next, explore Process Management to learn about job control, background processes, and signal handling!

Previous File Testing

Next Regular Expressions

Basic Directory Operations

Key Directory Commands

Recursive Directory Processing

Choosing the Right Method

Advanced find Operations

Essential find Expressions

find Actions

Practical Directory Patterns

Common Mistakes

1. Forgetting to quote glob patterns

2. Using semicolon without backslash

3. Iterating with ls output

Exercise: Directory Statistics

Summary

What's Next?

Enjoying these tutorials?