LISP - Read-Time Macros

Lisp - Reader Macros

A reader macro (or read macro) is a powerful feature that allows a programmer to extend Lisp’s own syntax. It modifies the behavior of the Lisp reader—the component that parses textual source code into Lisp objects. By defining a custom reader macro, you can create new, concise notations for your data structures or even embed domain-specific languages (DSLs) directly into your Lisp code.

Key Concepts: The Lisp Evaluation Pipeline

To understand reader macros, it’s crucial to know the stages a piece of Lisp code goes through:

• Read-Time: The Lisp reader consumes a stream of characters (from a file or the REPL) and produces Lisp objects (s-expressions). Reader macros operate at this stage.
• Compile-Time: The Lisp compiler takes the s-expressions produced by the reader. Regular macros (defined with defmacro) expand code at this stage.
• Run-Time: The final, compiled code is executed.

Purpose of Reader Macros

• Syntax Extension: Create custom syntactic sugar. The single quote (') is the most famous example: 'foo is read-time syntactic sugar for (quote foo).
• Custom Literals: Define literals for custom data types. For example, you could define a reader for regular expressions like #/\d+/.
• Read-Time Evaluation: Execute code during the reading process. The #. (hash-dot) macro does this, evaluating a Lisp form and inserting its result into the code being read.

How a Reader Macro Works

A reader macro is a function that you associate with one or more characters in the readtable. When the Lisp reader encounters a character that has a macro function attached, it calls that function. The function receives the input stream and the character as arguments. It is then responsible for parsing the input that follows and returning the Lisp object it represents.

Example: Re-implementing the Single Quote

The single quote (') is a built-in reader macro. Let’s examine how you would implement it yourself to understand the mechanism. This is a purely academic exercise, as you would never need to do this in practice.

;; 1. Define the function that will handle the macro character.
;;    It takes the stream and the character that triggered it as arguments.
(defun my-quote-reader (stream char)
  "Reads the next Lisp object and wraps it in a (quote ...) form."
  ;; We don't need the 'char' argument, so we ignore it.
  (declare (ignore char))
  ;; Construct the list: (quote <object>)
  (list 'quote (read stream t nil t)))

;; 2. Associate the function with a character in the readtable.
;;    Here, we (re)define the macro for the '#' character, followed by an apostrophe.
(set-macro-character #\' #'my-quote-reader)

;; 3. Now, we can test it.
;;    Typing \'foo into the REPL (or a file) will now be read as (quote foo).
;;    Let's verify what the reader produces:
(read-from-string "\'my-symbol")

Output

Executing (read-from-string "\'my-symbol") returns two values: the object read and the position of the next character in the string.

(QUOTE MY-SYMBOL) ; The Lisp object that was read
10               ; The position in the string after reading

Practical Applications of Reader Macros

• Custom Syntax: Create shorthand for common patterns. A project might define #{ ... } to read as a hash-set literal.
• Embedding Data Formats: Parse data formats like JSON or a simplified key: value pair notation directly into Lisp data structures at read time.
• Compile-Time Constants: The #. reader macro is useful for embedding the result of a calculation into compiled code, such as (defconstant +pi+ #.(- (acos 0) (acos 0))).

Common Errors and Debugging

Pitfall: Overriding standard characters like ( or " can break the Lisp reader entirely. It’s best practice to use dispatching macro characters (like #) to create your own unique syntax (e.g., #[, #<, etc.) to avoid conflicts.