Sed Pattern Buffer

Stream Editor - Understanding the Pattern Buffer

A core concept in SED is the pattern buffer. When SED processes input, it reads one line at a time (unless modified by commands like N) into this temporary storage area. Most SED commands operate on the text held in the pattern buffer. After all commands in the script have been applied to the line in the pattern buffer, SED typically prints the buffer’s contents to standard output and then reads the next line, repeating the cycle.

Let’s create a sample file named books.txt which we’ll use throughout these examples. Ensure your file content matches for consistent results.

File: `books.txt`

1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

The Print Command (p) and Default Output

The p command explicitly prints the current contents of the pattern buffer.

[jerry]$ sed 'p' books.txt

This command produces the following result:

1) A Storm of Swords, George R. R. Martin, 1216
1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864
6) A Game of Thrones, George R. R. Martin, 864

Each line appears twice because:

1. SED, by default, prints the pattern buffer after all commands for a line are executed.
2. We explicitly used the p command, which also prints the pattern buffer. To suppress the default printing action, use the -n (no-print) option.

[jerry]$ sed -n 'p' books.txt

Now, each line is printed only once, solely due to the p command:

1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

Addressing Lines for Operations

SED commands can be restricted to operate only on specific lines using addresses. An address can be a line number, a regular expression, or a range.

Print a specific line number (e.g., line 3):

[jerry]$ sed -n '3p' books.txt

Output:

3) The Alchemist, Paulo Coelho, 197

Print a range of lines (e.g., lines 2 through 5):

[jerry]$ sed -n '2,5p' books.txt

Output:

2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288

Special Addresses: $ (Last Line)

The $ character represents the last line of the input.

Print the last line:

[jerry]$ sed -n '$p' books.txt

Output:

6) A Game of Thrones, George R. R. Martin, 864

Print from line 3 to the last line:

[jerry]$ sed -n '3,$p' books.txt

Output:

3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864

Advanced Addressing (GNU SED Extensions)

GNU SED offers more flexible addressing schemes.

1. M,+n: Matches line M and the n lines immediately following it. So, it acts on n+1 lines in total.

Example: Print line 2 and the next 3 lines (total 4 lines: 2, 3, 4, 5).

[jerry]$ sed -n '2,+3p' books.txt

Output:

2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288

2. M~n: Matches line M and then every n-th line after line M.

Example: Print only odd-numbered lines (start at line 1, step by 2).

[jerry]$ sed -n '1~2p' books.txt

Output:

1) A Storm of Swords, George R. R. Martin, 1216
3) The Alchemist, Paulo Coelho, 197
5) The Pilgrimage, Paulo Coelho, 288

Example: Print only even-numbered lines (start at line 2, step by 2).

[jerry]$ sed -n '2~2p' books.txt

Output:

2) The Two Towers, J. R. R. Tolkien, 352
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
6) A Game of Thrones, George R. R. Martin, 864

Understanding the pattern buffer and addressing is fundamental to effectively using SED for text manipulation. Remember that these operations, by default, do not change the original file; they only affect the output stream. To modify the file in-place, use the -i option (e.g., sed -i 'command' file), but use it with caution, perhaps backing up your file first.