Caffe2 - Creating Your Own Network

PyTorch - Creating Your Own Network

In this lesson, you’ll learn how to define a simple single-layer neural network in PyTorch and perform a forward pass with sample data. We’ll cover PyTorch Tensors, Modules, and Linear layers. Basic familiarity with neural network concepts (layers, weights, biases, forward pass) is helpful.

PyTorch Tensors

The fundamental data structure in PyTorch is the Tensor (torch.Tensor), which is analogous to NumPy arrays but with added capabilities like GPU acceleration and automatic differentiation (Autograd).

Let’s create some tensors:

import torch
# Create a tensor with random values (e.g., for input data)
# Shape: [batch_size, num_features]
input_data = torch.randn(2, 3) # Batch of 2 samples, 3 features each
print("Input Data Tensor:\n", input_data)

# Create tensors for weights and biases (these are usually managed by nn.Module)
# Example shapes for a layer mapping 3 input features to 5 output features
weights = torch.randn(5, 3) # Shape: [output_features, input_features]
biases = torch.ones(5)      # Shape: [output_features]
print("\nExample Weights Tensor:\n", weights)
print("\nExample Biases Tensor:\n", biases)

Network Architecture: Single Linear Layer

We want to build a network representing a single fully connected (or ‘Linear’) layer. Mathematically, this computes: Y = X * W^T + b, where X is the input, W is the weight matrix, b is the bias vector, and Y is the output.

In PyTorch, network layers and models are typically defined by subclassing torch.nn.Module. This provides structure and helps manage parameters (like weights and biases).

Creating the Network using nn.Module

First, import the necessary components:

import torch.nn as nn

Now, define the network class:

class SingleLayerFC(nn.Module):
    def __init__(self, input_features, output_features):
        super(SingleLayerFC, self).__init__() # Call parent class constructor
        # Define the layer(s)
        self.linear_layer = nn.Linear(input_features, output_features)
        # nn.Linear automatically creates and initializes weight and bias tensors

    def forward(self, x):
        # Define the computation flow
        output = self.linear_layer(x)
        return output

Let’s instantiate this network. We need to specify the number of input features (matching our input_data) and the desired number of output features.

# Define dimensions
input_dim = 3
output_dim = 5

# Create an instance of the network
model = SingleLayerFC(input_dim, output_dim)

print("\nModel Structure:")
print(model)

Printing the model shows its structure, including the automatically created nn.Linear layer.

Running the Network (Forward Pass)

To get the output for our sample input_data, we simply call the model instance like a function:

# Pass the input data through the model
output_data = model(input_data)

print("\nInput Data Shape:", input_data.shape)
print("Output Data Tensor:\n", output_data)
print("Output Data Shape:", output_data.shape)

The output tensor output_data will have the shape [batch_size, output_features], which is [2, 5] in this case. The values are computed using the randomly initialized weights and biases within the nn.Linear layer.

Inspecting Parameters

We can examine the learnable parameters (weights and biases) that nn.Module automatically tracks:

print("\nModel Parameters:")
for name, param in model.named_parameters():
    if param.requires_grad:
        print(f"Name: {name}")
        print(f"Shape: {param.shape}")
        print(f"Values:\n{param.data}") # Use .data to view tensor values

This will show the ‘linear_layer.weight’ and ‘linear_layer.bias’ tensors managed by our model.

You have now successfully defined a simple neural network in PyTorch, passed data through it, and inspected its parameters. This forms the basis for building much more complex models.