Theano - Expression for Matrix Multiplication

Theano - Symbolic Expression for Matrix Multiplication

Let’s build upon the concept of symbolic variables to define a common operation: matrix multiplication (dot product). We’ll compute the dot product of two matrices, say matrix A (shape M x K) and matrix B (shape K x N), resulting in matrix C (shape M x N).

Remember, in Theano, we first define the symbolic computation, and then compile a function to execute it with actual data.

1. Import Necessary Libraries

import theano
import theano.tensor as tt
import numpy as np

2. Declaring Symbolic Variables

We declare two symbolic variables to represent our input matrices. We’ll use dmatrix, which signifies a matrix of double-precision floats (float64). Importantly, we don’t need to specify the exact dimensions (M, K, N) at this stage; Theano handles symbolic shapes.

# Declare symbolic matrix 'A' (float64)
a = tt.dmatrix('A')

# Declare symbolic matrix 'B' (float64)
b = tt.dmatrix('B')

print(f"Type of 'a': {a.type}")
print(f"Type of 'b': {b.type}")
# Output:
# Type of 'a': TensorType(float64, matrix)
# Type of 'b': TensorType(float64, matrix)

These variables a and b are just placeholders. They know they are matrices of floats, but they don’t hold specific numerical values yet.

3. Defining the Symbolic Expression

Now, we define the operation we want to perform using these symbolic variables. Theano provides the theano.tensor.dot() function for matrix multiplication (and dot products in general).

# Define the symbolic expression for the dot product
c = tt.dot(a, b)

# 'c' is also a symbolic variable representing the result.
# Theano infers its type based on the input types and the operation.
print(f"Type of 'c': {c.type}")
# Output:
# Type of 'c': TensorType(float64, matrix)

At this point, we have built a small computational graph: a and b are inputs, dot is the operation, and c is the output. No actual numbers have been crunched yet.

4. Compiling the Theano Function

To make this symbolic expression executable, we compile it into a Theano function. theano.function takes a list of symbolic inputs and a list of symbolic outputs.

# Compile the function
# Inputs: a list containing the symbolic variables 'a' and 'b'
# Outputs: a list containing the symbolic result 'c'
matrix_multiply = theano.function(inputs=[a, b], outputs=c)

print(f"Type of compiled function: {type(matrix_multiply)}")
# Output:
# Type of compiled function: <class 'theano.compile.function_module.Function'>

This compilation step is where Theano performs optimizations (graph simplification, potential C code generation, etc.) to make the execution efficient.

5. Invoking the Function with Data

Now we can use our compiled function matrix_multiply with actual numerical data. Theano functions typically expect NumPy arrays as input.

# Define concrete input matrices using NumPy
matrix_a = np.array([[0., -1., 2.], [4., 11., 2.]], dtype='float64') # Shape (2, 3)
matrix_b = np.array([[3., -1.], [1., 2.], [6., 1.]], dtype='float64') # Shape (3, 2)

print(f"Input Matrix A (shape {matrix_a.shape}):\n{matrix_a}")
print(f"Input Matrix B (shape {matrix_b.shape}):\n{matrix_b}")

# Call the compiled function with the NumPy arrays
result_matrix = matrix_multiply(matrix_a, matrix_b)

print(f"\nResult Matrix C (shape {result_matrix.shape}):\n{result_matrix}")
# Expected Output:
# Result Matrix C (shape (2, 2)):
# [[11.  0.]
#  [25. 20.]]

The function takes the NumPy arrays corresponding to the symbolic inputs (a, b) and computes the result corresponding to the symbolic output (c).

Full Program Listing

# Full program for matrix multiplication using Theano

import theano
import theano.tensor as tt
import numpy as np

# 1. Declare symbolic variables (placeholders for data)
a = tt.dmatrix('A') # Matrix of float64
b = tt.dmatrix('B') # Matrix of float64

# 2. Define the symbolic expression (the computation graph)
c = tt.dot(a, b)

# 3. Compile the expression into an executable function
matrix_multiply = theano.function(inputs=[a, b], outputs=c)

# 4. Prepare concrete input data (NumPy arrays)
matrix_a = np.array([[0., -1., 2.], [4., 11., 2.]], dtype='float64') # Shape (2, 3)
matrix_b = np.array([[3., -1.], [1., 2.], [6., 1.]], dtype='float64') # Shape (3, 2)

# 5. Invoke the function with the data
result_matrix = matrix_multiply(matrix_a, matrix_b)

# 6. Print the result
print(f"Input Matrix A:\n{matrix_a}")
print(f"Input Matrix B:\n{matrix_b}")
print(f"\nResult Matrix C (A dot B):\n{result_matrix}")

This example demonstrates the core workflow of Theano: define symbolic variables, build an expression graph, compile it into a function, and execute the function with concrete data (usually NumPy arrays).