Sed Managing Patterns

Advanced Pattern Space and Hold Space Management

Beyond basic line-by-line processing, sed offers powerful commands to manipulate its internal buffers: the pattern space and the hold space. Understanding these commands (n, N, h, H, g, G, x, p, P, d, D) unlocks sed’s ability to perform complex multi-line operations, text rearrangement, and conditional processing based on previous lines.

The n Command: Processing the Next Line

The lowercase n command fundamentally alters sed’s normal flow for the current line. Its actions are:

1. Print the current contents of the pattern space (unless default output is suppressed by -n or an earlier d command).
2. Clear the pattern space.
3. Read the next line of input into the now-empty pattern space. If there are no more input lines, sed terminates.
4. Execute any subsequent commands in the script on this new line in the pattern space.

Syntax:

[address1[,address2]]n

Example: Printing only even-numbered lines from books.txt (using the single-line-per-entry format):

$ sed -n 'n;p' books.txt

Output (assuming books.txt has at least 6 lines):

2) The Two Towers, J. R. R. Tolkien, 352
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
6) A Game of Thrones, George R. R. Martin, 864

Explanation:

• Line 1 read into pattern space. -n suppresses default print.
• n: Pattern space (Line 1) is not printed (due to -n). Next line (Line 2) is read into pattern space.
• p: Pattern space (Line 2) is printed.
• Line 3 read into pattern space.
• n: Pattern space (Line 3) is not printed. Next line (Line 4) is read into pattern space.
• p: Pattern space (Line 4) is printed. And so on.

Consider a script: command1; command2; n; command3; command4. command1 and command2 operate on the original line. Then n prints it (if not -n), reads the next line, and command3 and command4 operate on this newly read line.

Interacting with the Hold Space

The hold space is a secondary buffer that persists across sed’s read-execute-display cycles. It’s crucial for tasks requiring memory of previous lines. sed commands cannot directly operate on the hold space; data must be moved to the pattern space first. For the following examples, assume books_multiline.txt where each book’s title and author are on separate, consecutive lines:

A Storm of Swords
George R. R. Martin
The Two Towers
J. R. R. Tolkien
The Alchemist
Paulo Coelho
The Fellowship of the Ring
J. R. R. Tolkien
The Pilgrimage
Paulo Coelho
A Game of Thrones
George R. R. Martin

The x Command: Exchange Buffers

The x command swaps the contents of the pattern space and the hold space.

Example: Print only author names from books_multiline.txt.

$ sed -n 'x;n;p' books_multiline.txt

Output:

George R. R. Martin
J. R. R. Tolkien
Paulo Coelho
J. R. R. Tolkien
Paulo Coelho
George R. R. Martin

Workflow for the first two lines (Title1, Author1):

1. Line 1 (Title1) read into PS. PS={Title1}, HS={initially empty or content from previous cycle}.
2. x: Swaps. PS={HS_content}, HS={Title1}.
3. n: (No print due to -n). Reads Line 2 (Author1) into PS. PS={Author1}, HS={Title1}.
4. p: Prints PS, which is Author1.
5. This repeats. For the next pair, HS will contain Author1 initially, then Title2 after x.

The h and H Commands: Copying to Hold Space

h: Copies the pattern space to the hold space, overwriting the hold space’s previous contents.

H: Appends the pattern space (preceded by a newline) to the hold space.

Syntax: [address1[,address2]]h and [address1[,address2]]H

Example: Print only titles of books by “Paulo Coelho”.

$ sed -n '/Paulo Coelho/!h; /Paulo Coelho/{x;p;}' books_multiline.txt

Output:

The Alchemist
The Pilgrimage

Explanation:

• When a title line is read (e.g., “The Alchemist”), it does not match /Paulo Coelho/. So, h copies it to the hold space. HS={“The Alchemist”}.
• When the corresponding author line (“Paulo Coelho”) is read, it matches /Paulo Coelho/. The !h is skipped.
• The block {x;p;} executes: x swaps PS (“Paulo Coelho”) with HS (“The Alchemist”). Now PS={“The Alchemist”}, HS={“Paulo Coelho”}.
• p prints PS, which is the title “The Alchemist”.

Example: Print book titles followed by their authors for “Paulo Coelho”.

$ sed -n '/Paulo Coelho/!h; /Paulo Coelho/{H;x;p;}' books_multiline.txt

Output:

The Alchemist
Paulo Coelho
The Pilgrimage
Paulo Coelho

Explanation (for “The Alchemist” and “Paulo Coelho”):

• “The Alchemist” (title): h copies it to HS. HS={“The Alchemist”}.
• “Paulo Coelho” (author): Matches. H appends PS to HS. HS={“The Alchemist\nPaulo Coelho”}.
• x swaps. PS={“The Alchemist\nPaulo Coelho”}, HS={“Paulo Coelho”}.
• p prints PS.

The g and G Commands: Retrieving from Hold Space

g: Copies the hold space to the pattern space, overwriting the pattern space’s previous contents.

G: Appends the hold space (preceded by a newline) to the pattern space.

Syntax: [address1[,address2]]g and [address1[,address2]]G

Example: Print author “Paulo Coelho” then the corresponding book title.

$ sed -n '/Paulo Coelho/!h; /Paulo Coelho/{p;g;p;}' books_multiline.txt

Output:

Paulo Coelho
The Alchemist
Paulo Coelho
The Pilgrimage

Explanation (for “The Alchemist” and “Paulo Coelho”):

• “The Alchemist” (title): h copies it to HS. HS={“The Alchemist”}.
• “Paulo Coelho” (author): Matches. p prints PS (“Paulo Coelho”).
• g copies HS to PS. PS={“The Alchemist”}.
• p prints PS (“The Alchemist”).

Example using G: Print author “Paulo Coelho” then the corresponding book title (same result).

$ sed -n '/Paulo Coelho/!h; /Paulo Coelho/{G;p;}' books_multiline.txt

Explanation (for “The Alchemist” and “Paulo Coelho”):

• “The Alchemist” (title): h copies it to HS. HS={“The Alchemist”}.
• “Paulo Coelho” (author): Matches. G appends HS to PS. PS={“Paulo Coelho\nThe Alchemist”}.
• p prints PS.

Practical Applications:

These commands are the building blocks for complex text manipulations such as reversing the order of lines in a paragraph, processing blocks of text delimited by specific patterns, or collecting information from multiple lines to produce a summary. For example, one could collect all lines starting with ‘Error:’, store them in hold space, and print them all at the end of the file processing.

Common Pitfalls & Further Learning:

• Forgetting -n: If you’re manually controlling printing with p or P, remember to use -n to suppress automatic printing, or you’ll get duplicate lines.
• Newline character with H and G: These commands insert a newline before appending. This is usually helpful but needs to be accounted for in your patterns if you subsequently process the combined lines.
• Complexity: While powerful, deeply nested logic with pattern/hold space can become hard to debug. For very complex transformations, awk or scripting languages like Python or Perl might offer more clarity.
• Refer to the official GNU sed manual for authoritative details on each command and more examples.