R - XML Files

Modern XML Processing in R with `xml2`

XML (Extensible Markup Language) is a versatile format for structuring and sharing data across the web and other systems. Like HTML, it uses tags, but unlike HTML, which defines the presentation of content, XML tags define the meaning and structure of the data itself. This makes it a robust choice for data interchange and configuration files.

While the XML package has been a long-standing tool for this purpose, the modern R ecosystem, particularly the Tidyverse, favors the xml2 package. xml2 provides a more intuitive, efficient, and pipe-friendly interface, built on top of the robust libxml2 C library.

Prerequisites: Setting Up Your Environment

The xml2 package is a core part of the tidyverse, so you can install it by installing the entire suite. This also gives you powerful tools like dplyr and purrr which we will use for data manipulation.

# Best practice: Install the complete tidyverse for a cohesive workflow
install.packages("tidyverse")

# Load the necessary libraries
library(xml2)
library(dplyr) # For data manipulation, will be loaded with tidyverse
library(purrr) # For functional programming, also in tidyverse

Input Data: A Well-Formed XML Example

Let’s use a clean, well-formed XML file for our examples. Save the following content as employees.xml. Notice it has a single root element <records> and is not wrapped in <html> tags, which is standard for data files.

<?xml version="1.0" encoding="UTF-8"?>
<records>
  <employee id="101">
    <name>Rick</name>
    <salary>623.30</salary>
    <start_date>2012-01-01</start_date>
    <department>IT</department>
  </employee>
  <employee id="102">
    <name>Dan</name>
    <salary>515.20</salary>
    <start_date>2013-09-23</start_date>
    <department>Operations</department>
  </employee>
  <employee id="103">
    <name>Michelle</name>
    <salary>611.00</salary>
    <start_date>2014-11-15</start_date>
    <department>IT</department>
  </employee>
  <employee id="104">
    <name>Ryan</name>
    <salary>729.00</salary>
    <start_date>2014-05-11</start_date>
    <department>HR</department>
  </employee>
  <employee id="105">
    <name>Gary</name>
    <salary>843.25</salary>
    <start_date>2015-03-27</start_date>
    <department>Finance</department>
  </employee>
</records>

Reading and Inspecting XML Files

The read_xml() function parses the XML file into an R object. We can then use functions like xml_children() to inspect its contents.

# Read the XML file into an xml_document object
doc <- read_xml("employees.xml")

# Print the document to see its structure
print(doc)

When we execute the above code, it produces the following result:

{xml_document}
<records>
[1] <employee id="101">\n  <name>Rick</name>\n  <salary>623.3</salary>\n  <start_date>2012-01-01</start_date>\n  <department>IT</department>\n</employee>
[2] <employee id="102">\n  <name>Dan</name>\n  <salary>515.2</salary>\n  <start_date>2013-09-23</start_date>\n  <department>Operations</department>\n</employee>
[3] <employee id="103">\n  <name>Michelle</name>\n  <salary>611</salary>\n  <start_date>2014-11-15</start_date>\n  <department>IT</department>\n</employee>
[4] <employee id="104">\n  <name>Ryan</name>\n  <salary>729</salary>\n  <start_date>2014-05-11</start_date>\n  <department>HR</department>\n</employee>
[5] <employee id="105">\n  <name>Gary</name>\n  <salary>843.25</salary>\n  <start_date>2015-03-27</start_date>\n  <department>Finance</department>\n</employee>

Navigating XML with XPath

Instead of manual indexing, the modern and standard way to query XML is with XPath. The xml_find_all() function uses XPath expressions to select nodes. This is extremely powerful and precise.

# Find all 'employee' nodes under the root
employee_nodes <- xml_find_all(doc, ".//employee")

# Get the number of employee nodes found
num_employees <- length(employee_nodes)
cat("Number of employees found:", num_employees, "\n")

# Get the first employee node
first_employee <- employee_nodes[[1]]
print(first_employee)

# Extract specific information from the first employee
# Get the 'id' attribute
first_id <- xml_attr(first_employee, "id")
cat("First employee ID:", first_id, "\n")

# Find the 'name' tag within the first employee and get its text
first_name <- xml_find_first(first_employee, ".//name") |> xml_text()
cat("First employee Name:", first_name, "\n")

When we execute the above code, it produces the following result:

Number of employees found: 5
{xml_node}
<employee id="101">
[1] <name>Rick</name>
[2] <salary>623.3</salary>
[3] <start_date>2012-01-01</start_date>
[4] <department>IT</department>

First employee ID: 101
First employee Name: Rick

Best Practice: Convert XML to a Tidy Data Frame

For data analysis, the most useful format is a data frame (or a tibble). While the old XML package had xmlToDataFrame(), it was often unreliable with data types. The modern tidyverse approach is more explicit, robust, and readable. We will iterate over each <employee> node and extract its information into a single row.

# A helper function to process a single <employee> node
parse_employee_node <- function(node) {
  # Define a safe extraction function to handle missing tags
  safe_get_text <- function(n, xpath) {
    child <- xml_find_first(n, xpath)
    if (is.na(child)) NA_character_ else xml_text(child)
  }

  tibble(
    id = xml_attr(node, "id") |> as.integer(),
    name = safe_get_text(node, ".//name"),
    salary = safe_get_text(node, ".//salary") |> as.numeric(),
    start_date = safe_get_text(node, ".//start_date") |> as.Date(),
    department = safe_get_text(node, ".//department")
  )
}

# Find all employee nodes
employee_nodes <- xml_find_all(doc, ".//employee")

# Use purrr::map_dfr to apply the function to each node and row-bind the results
employees_df <- map_dfr(employee_nodes, parse_employee_node)

print(employees_df)

This method gives you full control over parsing, including handling missing values and setting correct data types. When we execute the above code, it produces a clean, tidy tibble:

# A tibble: 5 × 5
     id name     salary start_date department
  <int> <chr>     <dbl> <date>     <chr>
1   101 Rick       623. 2012-01-01 IT
2   102 Dan        515. 2013-09-23 Operations
3   103 Michelle   611  2014-11-15 IT
4   104 Ryan       729  2014-05-11 HR
5   105 Gary       843. 2015-03-27 Finance

Common Errors and Debugging

1. Malformed XML: read_xml() will throw an error if the XML is not well-formed. Use an online validator to check your file.

2. Missing Nodes/Attributes: If xml_find_first() doesn’t find a node, it returns NA. If you try to use xml_text() or xml_attr() on NA, you will get an error. Our safe_get_text helper function demonstrates one way to handle this gracefully.

3. Namespaces: Real-world XML often uses namespaces (e.g., <ns1:tag>). To query these, you must register the namespace with xml_ns() and use it in your XPath queries. This is an advanced topic but a common stumbling block.