Theano - Data Types

Theano - Data Types and Symbolic Variables

Before diving into building models, it’s essential to understand how Theano represents data. Theano works with symbolic variables, which are placeholders for actual data. You define the structure and type of these variables first, then build expressions using them. Actual numerical computation happens later when you compile and run a Theano function.

Theano provides various types to define these symbolic variables, primarily focusing on tensors (multi-dimensional arrays), similar to NumPy. Here’s a summary of common Theano tensor types:

Concept	Common Theano Types (using theano.tensor)	Description
Scalar (0D Tensor)	scalar (generic), bscalar, wscalar, iscalar, lscalar, fscalar, dscalar, cscalar	A single value (e.g., integer, float). Prefixes indicate dtype: b=byte, w=int16, i=int32, l=int64, f=float32, d=float64, c=complex64/128.
Vector (1D Tensor)	vector (generic), bvector, wvector, ivector, lvector, fvector, dvector, cvector	A one-dimensional array.
Matrix (2D Tensor)	matrix (generic), bmatrix, wmatrix, imatrix, lmatrix, fmatrix, dmatrix, cmatrix	A two-dimensional array (rows and columns).
Row Vector (Special Matrix)	row (generic), brow, wrow, irow, lrow, frow, drow, crow	A matrix guaranteed to have exactly one row.
Column Vector (Special Matrix)	col (generic), bcol, wcol, icol, lcol, fcol, dcol, ccol	A matrix guaranteed to have exactly one column.
3D Tensor	tensor3 (generic), btensor3, wtensor3, itensor3, ltensor3, ftensor3, dtensor3, ctensor3	A three-dimensional array.
ND Tensor (N > 3)	tensor4, tensor5, etc. (generic), btensor4, …, dtensor4, …	Higher-dimensional arrays.

Note: While Theano offered highly specific types (like wscalar for int16), modern libraries often default to standard types like int32, int64, float32, and float64. The f (float32) and d (float64) types were very common in Theano for machine learning.

Let’s see how to declare some of these symbolic variables. It’s good practice to import theano.tensor with an alias like tt.

import theano
import theano.tensor as tt
import numpy as np

Declaring a Scalar

To declare a symbolic scalar variable (representing a single floating-point number):

# Declare a symbolic scalar variable named 'x' of type float64
x = tt.dscalar('x')

# At this point, 'x' is just a symbol. It doesn't hold a value yet.
print(type(x))
# Output will show something like: <class 'theano.tensor.var.TensorVariable'>

You assign a concrete value later when you run a Theano function involving this variable.

Declaring a Vector (1D Array)

To declare a symbolic vector (1D array) of float32 values:

# Declare a symbolic vector named 'my_vector' of type float32
f_vec = tt.fvector('my_vector')

# Again, this is just a placeholder.
print(f_vec.type)
# Output: TensorType(float32, vector)

You can create expressions using these symbolic variables:

# Example expression: element-wise square of the vector
f_vec_squared = f_vec ** 2

# To actually compute this for specific values, you need a Theano function:
# (Theano functions are covered in detail later)
calculate_square = theano.function(inputs=[f_vec], outputs=f_vec_squared)

# Now, call the function with actual data (a NumPy array)
result = calculate_square(np.array([1.0, 2.0, 3.0], dtype='float32'))
print(result)
# Output: [1. 4. 9.]

Declaring a Matrix (2D Array)

To declare a symbolic matrix (2D array) of integers:

# Declare a symbolic matrix named 'int_matrix' of type int32
i_mat = tt.imatrix('int_matrix')

print(i_mat.type)
# Output: TensorType(int32, matrix)

Let’s define a simple operation:

# Expression: Transpose the matrix
i_mat_transposed = i_mat.T

# Compile a function to compute the transpose
get_transpose = theano.function(inputs=[i_mat], outputs=i_mat_transposed)

# Execute with data
data = np.array([[1, 2, 3], [4, 5, 6]], dtype='int32')
transposed_result = get_transpose(data)
print(transposed_result)
# Output:
# [[1 4]
#  [2 5]
#  [3 6]]

Declaring Higher-Dimensional Tensors

You can declare tensors of higher dimensions similarly:

# Declare a 4D tensor of float64 values
tensor4d = tt.dtensor4('my_4d_tensor')

print(tensor4d.type)
# Output: TensorType(float64, 4D)

These are essential for representing data like batches of images (Batch, Height, Width, Channels) or sequences in deep learning.

Generic Constructors and Explicit dtypes

Instead of specific types like fvector or dmatrix, you can use generic constructors like scalar, vector, matrix, tensor3, tensor4 and specify the dtype explicitly. This aligns more closely with NumPy and modern libraries.

# Declare a float32 vector using the generic constructor
generic_vec = tt.vector('generic_vec', dtype='float32')
print(generic_vec.type)
# Output: TensorType(float32, vector)

# Declare an int64 matrix
generic_mat = tt.matrix('generic_mat', dtype='int64')
print(generic_mat.type)
# Output: TensorType(int64, matrix)

Using these generic constructors with explicit dtype is often clearer and more flexible.

Important Note: Theano’s Legacy

While understanding Theano’s data types and symbolic nature is valuable for historical context and grasping concepts like computational graphs, Theano itself is no longer actively developed. Modern deep learning frameworks like TensorFlow, PyTorch, and JAX offer similar tensor functionalities with more streamlined APIs and better performance optimizations for current hardware. The concepts learned here, however, remain relevant.