Iterators in Lua

Intermediate ~30 min read

Iterators are functions that allow you to traverse collections in a controlled way. Lua's generic for loop works with iterators to provide elegant iteration patterns. Understanding how iterators work enables you to create custom iteration logic for any data structure. Let's explore how to create and use iterators effectively!

The Iterator Protocol

The generic for loop expects an iterator function:

for var1, var2, ... in iterator do
    -- body
end

The iterator function is called repeatedly, returning values until it returns nil:

-- Simple iterator
local function simpleIterator()
    local i = 0
    return function()
        i = i + 1
        if i <= 3 then
            return i
        end
    end
end

for value in simpleIterator() do
    print(value)  -- 1, 2, 3
end

Output

Click Run to execute your code

Stateless Iterators

Stateless iterators don't maintain state between calls. They receive the state from the for loop:

-- Iterator function
local function iter(t, i)
    i = i + 1
    local v = t[i]
    if v then
        return i, v
    end
end

-- Factory function
local function values(t)
    return iter, t, 0
end

-- Usage
local fruits = {"apple", "banana", "cherry"}
for i, v in values(fruits) do
    print(i, v)
end

How it works:

Factory returns: iterator function, invariant state, control variable
For loop calls: iterator(state, control)
Iterator returns new control variable and values
Loop continues until iterator returns nil

ipairs Implementation

Here's how ipairs works internally:

local function myIpairs(t)
    local function iter(t, i)
        i = i + 1
        local v = t[i]
        if v ~= nil then
            return i, v
        end
    end
    return iter, t, 0
end

local arr = {10, 20, 30}
for i, v in myIpairs(arr) do
    print(i, v)
end

-- Stateless Iterator Examples in Lua

-- Custom stateless iterator for a range
local function range_iter(max, current)
    current = current + 1
    if current <= max then
        return current
    end
end

function range(max)
    return range_iter, max, 0
end

print("=== Custom range iterator ===")
for i in range(5) do
    print(i)
end

-- Stateless iterator for even numbers
local function even_iter(max, current)
    current = current + 2
    if current <= max then
        return current
    end
end

function evens(max)
    return even_iter, max, 0
end

print("\n=== Even numbers ===")
for num in evens(10) do
    print(num)
end

-- Stateless iterator for array with filter
local function filter_iter(t, i)
    i = i + 1
    local v = t[i]
    if v then
        if v > 10 then
            return i, v
        else
            return filter_iter(t, i)
        end
    end
end

function filter_greater_than_10(t)
    return filter_iter, t, 0
end

print("\n=== Filtered values > 10 ===")
local numbers = {5, 15, 8, 20, 12, 3}
for i, v in filter_greater_than_10(numbers) do
    print(i, v)
end

Output

Click Run to execute your code

Stateful Iterators

Stateful iterators use closures to maintain state:

local function range(from, to, step)
    step = step or 1
    local current = from - step
    
    return function()
        current = current + step
        if current <= to then
            return current
        end
    end
end

-- Usage
for i in range(1, 10, 2) do
    print(i)  -- 1, 3, 5, 7, 9
end

Fibonacci Iterator

local function fibonacci(n)
    local a, b = 0, 1
    local count = 0
    
    return function()
        if count < n then
            count = count + 1
            a, b = b, a + b
            return a
        end
    end
end

for num in fibonacci(10) do
    print(num)  -- 1, 1, 2, 3, 5, 8, 13, 21, 34, 55
end

-- Stateful Iterator Examples in Lua

-- Stateful iterator using closure
function file_lines(filename)
    local file = io.open(filename)
    return function()
        if file then
            local line = file:read()
            if line then
                return line
            else
                file:close()
                file = nil
            end
        end
    end
end

-- Stateful iterator for Fibonacci sequence
function fibonacci()
    local a, b = 0, 1
    return function()
        a, b = b, a + b
        return a
    end
end

print("=== Fibonacci sequence (first 10) ===")
local fib = fibonacci()
for i = 1, 10 do
    print(fib())
end

-- Stateful iterator with state object
function counter(start, step)
    local state = {
        current = start,
        step = step or 1
    }
    
    return function()
        local value = state.current
        state.current = state.current + state.step
        return value
    end
end

print("\n=== Counter from 10, step 3 ===")
local count = counter(10, 3)
for i = 1, 5 do
    print(count())
end

-- Stateful iterator for unique values
function unique_iterator(t)
    local seen = {}
    local i = 0
    
    return function()
        i = i + 1
        while t[i] do
            if not seen[t[i]] then
                seen[t[i]] = true
                return t[i]
            end
            i = i + 1
        end
    end
end

print("\n=== Unique values ===")
local items = {1, 2, 2, 3, 1, 4, 3, 5}
for value in unique_iterator(items) do
    print(value)
end

Output

Click Run to execute your code

Custom Iterators

Words Iterator

local function words(str)
    local pos = 1
    return function()
        local start, finish = str:find("%w+", pos)
        if start then
            pos = finish + 1
            return str:sub(start, finish)
        end
    end
end

local text = "Hello World from Lua"
for word in words(text) do
    print(word)
end

Lines Iterator

local function lines(str)
    local pos = 1
    return function()
        if pos > #str then return nil end
        local start = pos
        local finish = str:find("\n", pos) or #str + 1
        pos = finish + 1
        return str:sub(start, finish - 1)
    end
end

local text = "Line 1\nLine 2\nLine 3"
for line in lines(text) do
    print(line)
end

Reverse Iterator

local function reverse(t)
    local i = #t + 1
    return function()
        i = i - 1
        if i > 0 then
            return i, t[i]
        end
    end
end

local arr = {10, 20, 30, 40}
for i, v in reverse(arr) do
    print(i, v)  -- 4,40  3,30  2,20  1,10
end

-- Custom Iterator Examples in Lua

-- Custom iterator for key-value pairs in reverse order
function reverse_pairs(t)
    local keys = {}
    for k in pairs(t) do
        table.insert(keys, k)
    end
    
    local i = #keys + 1
    return function()
        i = i - 1
        if i > 0 then
            local k = keys[i]
            return k, t[k]
        end
    end
end

print("=== Reverse iteration ===")
local data = {a = 1, b = 2, c = 3, d = 4}
for k, v in reverse_pairs(data) do
    print(k, v)
end

-- Custom iterator for array in reverse
function reverse_ipairs(t)
    local i = #t + 1
    return function()
        i = i - 1
        if i > 0 then
            return i, t[i]
        end
    end
end

print("\n=== Reverse array ===")
local arr = {10, 20, 30, 40, 50}
for i, v in reverse_ipairs(arr) do
    print(i, v)
end

-- Custom iterator with step
function step_iterator(t, step)
    step = step or 1
    local i = 0
    return function()
        i = i + step
        if t[i] then
            return i, t[i]
        end
    end
end

print("\n=== Every 2nd element ===")
local numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
for i, v in step_iterator(numbers, 2) do
    print(i, v)
end

-- Custom iterator for tree traversal
function tree_iterator(tree)
    local stack = {tree}
    
    return function()
        if #stack > 0 then
            local node = table.remove(stack)
            if node.children then
                for i = #node.children, 1, -1 do
                    table.insert(stack, node.children[i])
                end
            end
            return node.value
        end
    end
end

print("\n=== Tree traversal ===")
local tree = {
    value = "root",
    children = {
        {value = "child1"},
        {value = "child2", children = {
            {value = "grandchild1"},
            {value = "grandchild2"}
        }}
    }
}

for value in tree_iterator(tree) do
    print(value)
end

Output

Click Run to execute your code

Advanced Iterator Patterns

Filter Iterator

local function filter(predicate, iterator, state, var)
    return function()
        while true do
            local value
            var, value = iterator(state, var)
            if var == nil then return nil end
            if predicate(value) then
                return var, value
            end
        end
    end, state, var
end

-- Usage
local numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
for i, v in filter(function(x) return x % 2 == 0 end, ipairs(numbers)) do
    print(i, v)  -- Even numbers only
end

Map Iterator

local function map(func, iterator, state, var)
    return function()
        local value
        var, value = iterator(state, var)
        if var == nil then return nil end
        return var, func(value)
    end, state, var
end

-- Usage
local numbers = {1, 2, 3, 4, 5}
for i, v in map(function(x) return x * 2 end, ipairs(numbers)) do
    print(i, v)  -- Doubled values
end

Take Iterator

local function take(n, iterator, state, var)
    local count = 0
    return function()
        if count >= n then return nil end
        count = count + 1
        return iterator(state, var)
    end, state, var
end

-- Usage
local function infiniteOnes()
    return function() return 1 end
end

for value in take(5, infiniteOnes()) do
    print(value)  -- 1, 1, 1, 1, 1
end

-- Advanced Iterator Techniques in Lua

-- Chaining iterators
function map(func, iter, state, var)
    return function()
        local values = {iter(state, var)}
        var = values[1]
        if var ~= nil then
            return var, func(values[2])
        end
    end
end

function filter(pred, iter, state, var)
    return function()
        while true do
            local values = {iter(state, var)}
            var = values[1]
            if var == nil or pred(values[2]) then
                return var, values[2]
            end
        end
    end
end

print("=== Map and Filter ===")
local numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

-- Filter even numbers and multiply by 2
local function is_even(n) return n % 2 == 0 end
local function double(n) return n * 2 end

for i, v in filter(is_even, ipairs(numbers)) do
    print(i, v)
end

-- Iterator composition
function compose(...)
    local funcs = {...}
    return function(...)
        local result = {...}
        for i = #funcs, 1, -1 do
            result = {funcs[i](unpack(result))}
        end
        return unpack(result)
    end
end

-- Lazy evaluation iterator
function lazy_map(func, t)
    local i = 0
    return function()
        i = i + 1
        if t[i] then
            return i, func(t[i])
        end
    end
end

print("\n=== Lazy map (square) ===")
local nums = {1, 2, 3, 4, 5}
for i, v in lazy_map(function(x) return x * x end, nums) do
    print(i, v)
end

-- Take first N elements
function take(n, iter, state, var)
    local count = 0
    return function()
        if count < n then
            count = count + 1
            local values = {iter(state, var)}
            var = values[1]
            if var ~= nil then
                return unpack(values)
            end
        end
    end
end

print("\n=== Take first 3 ===")
local all_nums = {10, 20, 30, 40, 50, 60}
for i, v in take(3, ipairs(all_nums)) do
    print(i, v)
end

-- Zip two iterators
function zip(t1, t2)
    local i = 0
    return function()
        i = i + 1
        if t1[i] and t2[i] then
            return i, t1[i], t2[i]
        end
    end
end

print("\n=== Zip two arrays ===")
local names = {"Alice", "Bob", "Charlie"}
local ages = {25, 30, 35}
for i, name, age in zip(names, ages) do
    print(i, name, age)
end

Output

Click Run to execute your code

Practical Examples

Tree Traversal

local function traverse(node)
    local stack = {node}
    
    return function()
        if #stack == 0 then return nil end
        
        local current = table.remove(stack)
        
        -- Add children to stack
        if current.children then
            for i = #current.children, 1, -1 do
                table.insert(stack, current.children[i])
            end
        end
        
        return current
    end
end

local tree = {
    value = 1,
    children = {
        {value = 2},
        {value = 3, children = {{value = 4}, {value = 5}}}
    }
}

for node in traverse(tree) do
    print(node.value)
end

Pagination Iterator

local function paginate(items, pageSize)
    local page = 0
    local totalPages = math.ceil(#items / pageSize)
    
    return function()
        if page >= totalPages then return nil end
        page = page + 1
        
        local start = (page - 1) * pageSize + 1
        local finish = math.min(page * pageSize, #items)
        
        local pageItems = {}
        for i = start, finish do
            table.insert(pageItems, items[i])
        end
        
        return page, pageItems
    end
end

local items = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
for page, pageItems in paginate(items, 3) do
    print("Page " .. page .. ":", table.concat(pageItems, ", "))
end

-- Practical Iterator Examples in Lua

-- CSV parser iterator
function csv_iterator(str)
    local pos = 1
    return function()
        if pos <= #str then
            local line_end = str:find("\n", pos) or #str + 1
            local line = str:sub(pos, line_end - 1)
            pos = line_end + 1
            
            local fields = {}
            for field in line:gmatch("[^,]+") do
                table.insert(fields, field)
            end
            return fields
        end
    end
end

print("=== CSV Parser ===")
local csv_data = "Alice,25,Engineer\nBob,30,Designer\nCharlie,35,Manager"
for fields in csv_iterator(csv_data) do
    print(table.concat(fields, " | "))
end

-- Word iterator
function words(str)
    local pos = 1
    return function()
        local word_start, word_end = str:find("%w+", pos)
        if word_start then
            pos = word_end + 1
            return str:sub(word_start, word_end)
        end
    end
end

print("\n=== Word iterator ===")
local text = "Hello, world! This is Lua."
for word in words(text) do
    print(word)
end

-- Batch iterator (process items in batches)
function batch_iterator(t, batch_size)
    local i = 0
    return function()
        if i < #t then
            local batch = {}
            for j = 1, batch_size do
                i = i + 1
                if t[i] then
                    table.insert(batch, t[i])
                else
                    break
                end
            end
            if #batch > 0 then
                return batch
            end
        end
    end
end

print("\n=== Batch processing (size 3) ===")
local items = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
for batch in batch_iterator(items, 3) do
    print("Batch:", table.concat(batch, ", "))
end

-- Sliding window iterator
function sliding_window(t, size)
    local i = 0
    return function()
        i = i + 1
        if i + size - 1 <= #t then
            local window = {}
            for j = i, i + size - 1 do
                table.insert(window, t[j])
            end
            return window
        end
    end
end

print("\n=== Sliding window (size 3) ===")
local sequence = {1, 2, 3, 4, 5, 6}
for window in sliding_window(sequence, 3) do
    print("Window:", table.concat(window, ", "))
end

-- Enumerate iterator (like Python's enumerate)
function enumerate(t, start)
    start = start or 1
    local i = start - 1
    return function()
        i = i + 1
        if t[i - start + 1] then
            return i, t[i - start + 1]
        end
    end
end

print("\n=== Enumerate (start at 10) ===")
local colors = {"red", "green", "blue"}
for index, color in enumerate(colors, 10) do
    print(index, color)
end

Output

Click Run to execute your code

Practice Exercise

Try these iterator challenges:

-- Iterator Exercises

-- Exercise 1: Create a custom iterator that returns only odd numbers from a table
function odd_numbers(t)
    -- TODO: Implement iterator for odd numbers
    local i = 0
    return function()
        i = i + 1
        while t[i] do
            if t[i] % 2 ~= 0 then
                return i, t[i]
            end
            i = i + 1
        end
    end
end

-- Test Exercise 1
print("=== Exercise 1: Odd numbers ===")
local nums = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
for i, v in odd_numbers(nums) do
    print(i, v)
end

-- Exercise 2: Create an iterator that skips nil values
function skip_nil(t)
    -- TODO: Implement iterator that skips nil values
    local i = 0
    return function()
        i = i + 1
        while i <= #t + 10 do  -- Check beyond array length
            if t[i] ~= nil then
                return i, t[i]
            end
            i = i + 1
        end
    end
end

-- Test Exercise 2
print("\n=== Exercise 2: Skip nil values ===")
local sparse = {10, nil, 30, nil, 50}
for i, v in skip_nil(sparse) do
    print(i, v)
end

-- Exercise 3: Create a pairwise iterator (returns consecutive pairs)
function pairwise(t)
    -- TODO: Implement pairwise iterator
    local i = 0
    return function()
        i = i + 1
        if t[i] and t[i + 1] then
            return i, t[i], t[i + 1]
        end
    end
end

-- Test Exercise 3
print("\n=== Exercise 3: Pairwise ===")
local items = {1, 2, 3, 4, 5}
for i, a, b in pairwise(items) do
    print(i, a, b)
end

-- Exercise 4: Create a cycle iterator (repeats the sequence)
function cycle(t, times)
    -- TODO: Implement cycle iterator
    local i = 0
    local count = 0
    return function()
        i = i + 1
        if i > #t then
            i = 1
            count = count + 1
        end
        if count < times then
            return i, t[i]
        end
    end
end

-- Test Exercise 4
print("\n=== Exercise 4: Cycle 3 times ===")
local pattern = {"A", "B", "C"}
for i, v in cycle(pattern, 3) do
    print(i, v)
end

-- Exercise 5: Create a flatten iterator for nested tables
function flatten(t)
    -- TODO: Implement flatten iterator
    local stack = {{table = t, index = 1}}
    
    return function()
        while #stack > 0 do
            local current = stack[#stack]
            local value = current.table[current.index]
            
            if value == nil then
                table.remove(stack)
            elseif type(value) == "table" then
                current.index = current.index + 1
                table.insert(stack, {table = value, index = 1})
            else
                current.index = current.index + 1
                return value
            end
        end
    end
end

-- Test Exercise 5
print("\n=== Exercise 5: Flatten nested tables ===")
local nested = {1, {2, 3}, {4, {5, 6}}, 7}
for v in flatten(nested) do
    print(v)
end

Output

Click Run to execute your code

Summary

In this lesson, you learned:

The iterator protocol and how generic for loops work
Stateless iterators that receive state from the loop
Stateful iterators using closures
Custom iterators for specific use cases
Advanced patterns: filter, map, take
Practical examples: tree traversal, pagination

What's Next?

Congratulations on completing Module 4! You've mastered tables, the table library, metatables, and iterators—the core of Lua's data manipulation capabilities. Next, we'll explore Object-Oriented Programming in Lua. You'll learn how to create classes, implement inheritance, and build robust OOP systems using tables and metatables. Let's continue! 🚀

Previous Metatables

Next OOP Basics

The Iterator Protocol

Stateless Iterators

ipairs Implementation

Stateful Iterators

Fibonacci Iterator

Custom Iterators

Words Iterator

Lines Iterator

Reverse Iterator

Advanced Iterator Patterns

Filter Iterator

Map Iterator

Take Iterator

Practical Examples

Tree Traversal

Pagination Iterator

Practice Exercise

Summary

What's Next?

Enjoying these tutorials?