YAML – Processes

YAML - Data Processing Workflow

YAML facilitates the conversion of data between its human-readable text format and native data structures used by programming languages. This process involves several key stages, whether converting application data into YAML (serialization/presentation) or YAML text back into application data (parsing/representation).

The YAML Processing Model

The general flow can be visualized as follows:

From Application Data to YAML Text:

1. Application’s Native Data Structures (e.g., objects, arrays, lists, dictionaries in Python, Java, JavaScript).
2. YAML Representation (Node Graph): This native data is transformed into an abstract, internal YAML model consisting of nodes (scalars, sequences, mappings), often called a node graph or representation graph.
3. Serialization (Event Stream): The node graph is processed to generate a sequence of abstract YAML events (e.g., ‘start sequence’, ‘scalar value’, ‘end mapping’).
4. Presentation (Character Stream): These events are then formatted into the final human-readable YAML text, including indentation, comments, and styling choices.

From YAML Text to Application Data:

1. YAML Character Stream (Input: the .yaml/.yml file content).
2. Parsing (Event Stream): The YAML text is read and parsed into a sequence of YAML events, validating syntax along the way.
3. YAML Representation (Node Graph): These events are used to construct the abstract YAML node graph in memory.
4. Reconstruction into Native Data Structures: The node graph is then converted back into the programming language’s native data types.

YAML processors (libraries in various programming languages) handle these conversions. Let’s look at the key stages:

1. Representation

At its core, YAML represents data using three fundamental node types:

Sequences

Represent ordered collections of items, equivalent to arrays or lists. Example:

tasks:
  - sku: BL394D
    description: Football
    quantity: 4
  - sku: BL4438H
    description: Super Hoop
    quantity: 1

Mappings

Represent unordered collections of key-value pairs, like dictionaries, hash tables, or objects. Keys must be unique. Example:

database_config:
  host: localhost
  port: 5432
  user: admin

Scalars

Represent individual, atomic data values like strings, numbers (integers, floats), booleans, nulls, dates, etc. Examples: BL394D, 4, localhost, 5432, true.

2. Serialization

Serialization is the process of converting an application’s in-memory data structures (the YAML representation/node graph) into a YAML event stream, which then leads to the textual YAML format. This stage considers aspects like key order (though mappings are inherently unordered, some serializers preserve insertion order for human-friendliness) and resolving anchors/aliases.

For example, a Python dictionary might be serialized into a YAML mapping:

# Python dictionary (conceptual)
# my_data = {'name': 'MyApp', 'version': '1.0', 'settings': {'debug': True, 'port': 8080}}

# Serialized YAML Output
name: MyApp
version: '1.0'
settings:
  debug: true
  port: 8080

3. Presentation

Presentation is the final stage of generating the YAML character stream (the actual text file). The YAML processor makes decisions about formatting, such as indentation width, whether to use block or flow style for collections, how to quote scalars, and where to place comments if they were part of the representation. User preferences can often guide this process.

The output is human-readable YAML, like the examples shown above. If the target is JSON, the presentation would conform to JSON syntax.

4. Parsing

Parsing is the inverse of serialization and presentation. It takes a YAML character stream as input and performs lexical and syntactical analysis to ensure it’s well-formed according to YAML specifications. If errors are found (e.g., inconsistent indentation, syntax violations), the parser will report them. A successful parse generates an event stream, which is then used to build the YAML representation (node graph) in memory.

Consider this YAML input:

---
environment: production
application_nodes:
  - id: app_server_01
    role: web_api
  - id: db_server_01
    role: database
parameters:
  max_connections: 100

A YAML parser reads this, validates it, and allows an application to access environment as ‘production’, application_nodes as a list of objects, etc.

Validation Tools

Online tools like YAML Lint (e.g., accessible via sites like yamllint.com or integrated into IDEs) can be used to parse and validate YAML documents. These tools check for syntactical correctness and can help identify issues before the YAML is used by an application. Many code editors and IDEs also offer real-time YAML validation.

Understanding these processes helps in effectively using YAML for configuration, data interchange, and appreciating how YAML libraries work under the hood.