Implementing First Neural Network

PyTorch - Implementing Your First Neural Network

PyTorch excels at creating and training neural networks. In this chapter, we’ll guide you through building a simple neural network with one hidden layer, designed for a binary classification task (predicting one of two outcomes).

We’ll follow these steps to implement our first neural network using PyTorch:

Step 1: Import Necessary Libraries

First, we need to import the PyTorch library and its neural network module (nn).

import torch
import torch.nn as nn

It’s also good practice to set up the device (CPU or GPU) for computation early on. This makes your code portable.

# Determine if a GPU is available, otherwise use CPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f'Using device: {device}')

Step 2: Define Network Architecture and Hyperparameters

Next, we define the sizes of our input layer, hidden layer, and output layer, as well as the batch size for training.

# Define network parameters
n_in, n_h, n_out, batch_size = 10, 5, 1, 10 # input features, hidden units, output units, batch size

Step 3: Create Dummy Data

Neural networks learn by mapping input data to corresponding target outputs. We’ll create some random input data (x) and binary target data (y).

# Create dummy input and target tensors (data)
# Input data: random values, shape (batch_size, n_in)
x = torch.randn(batch_size, n_in, device=device)

# Target data: binary values (0 or 1), shape (batch_size, n_out)
# For a binary classification task, targets should be floats for loss calculation.
y = torch.tensor([
    [1.0],
    [0.0],
    [0.0],
    [1.0],
    [1.0],
    [1.0],
    [0.0],
    [0.0],
    [1.0],
    [1.0]
], device=device)

Note: We move our data to the chosen device to ensure computations happen where intended.

Step 4: Define the Model

We’ll use nn.Sequential to create a simple feedforward network. It consists of a linear layer, a ReLU activation function, another linear layer, and no final activation here because we’ll use BCEWithLogitsLoss which is more numerically stable.

# Create a sequential model
model = nn.Sequential(
    nn.Linear(n_in, n_h),  # First linear layer (input to hidden)
    nn.ReLU(),             # ReLU activation function
    nn.Linear(n_h, n_out)  # Second linear layer (hidden to output)
).to(device) # Move the model to the selected device

print("Model architecture:")
print(model)

Using nn.Linear(n_h, n_out) without a final nn.Sigmoid() is recommended when using nn.BCEWithLogitsLoss, as it combines the Sigmoid activation and Binary Cross Entropy loss in a more numerically stable way.

Step 5: Define Loss Function and Optimizer

For binary classification, nn.BCEWithLogitsLoss is a suitable loss function. We’ll use Stochastic Gradient Descent (SGD) as our optimizer. Adam is another popular and often effective choice.

# Construct the loss function
# BCEWithLogitsLoss is suitable for binary classification and expects raw logits from the model.
criterion = nn.BCEWithLogitsLoss()

# Construct the optimizer (Stochastic Gradient Descent in this case)
# Adam is also a very popular and effective optimizer: torch.optim.Adam(model.parameters(), lr=0.01)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

Step 6: Train the Model (Training Loop)

The training process involves iterating multiple times (epochs) over the data, performing a forward pass, calculating the loss, performing a backward pass (backpropagation), and updating the model’s weights.

# Training loop (Gradient Descent)
epochs = 100 # Increased epochs for better convergence demonstration
print("\nStarting training...")
for epoch in range(epochs):
    # Forward pass: Compute predicted y by passing x to the model
    y_pred_logits = model(x)

    # Compute and print loss
    # The y_pred_logits are raw scores (logits), criterion handles the sigmoid activation implicitly.
    loss = criterion(y_pred_logits, y)
    if (epoch + 1) % 10 == 0 or epoch == 0: # Print every 10 epochs and the first one
        print(f'Epoch: {epoch+1}/{epochs}, Loss: {loss.item():.4f}')

    # Zero gradients: Clear previous gradients before computing new ones.
    # If you don't zero the gradients, they will accumulate, which is usually not desired.
    optimizer.zero_grad()

    # Backward pass: Compute gradients of the loss with respect to model parameters.
    loss.backward()

    # Update the weights: Adjust model parameters based on the computed gradients.
    optimizer.step()

print("Training finished.")

Key steps in the training loop:

• Forward Pass: Input data is fed through the network to get predictions.
• Loss Calculation: The difference between predictions and actual targets is quantified.
• Zero Gradients: Gradients from the previous iteration are cleared.
• Backward Pass (Backpropagation): Gradients of the loss with respect to each model parameter are calculated.
• Optimizer Step: Model parameters (weights) are updated using the calculated gradients to minimize the loss.

Step 7: Observing the Output

During training, you’ll see the loss generally decreasing, indicating that the model is learning. The exact loss values will vary due to random initialization.

# Example Output (values will differ due to random initialization):
# Using device: cuda (or cpu)
# Model architecture:
# Sequential(
#   (0): Linear(in_features=10, out_features=5, bias=True)
#   (1): ReLU()
#   (2): Linear(in_features=5, out_features=1, bias=True)
# )
#
# Starting training...
# Epoch: 1/100, Loss: 0.7512
# Epoch: 10/100, Loss: 0.7225
# Epoch: 20/100, Loss: 0.6953
# Epoch: 30/100, Loss: 0.6698
# Epoch: 40/100, Loss: 0.6459
# Epoch: 50/100, Loss: 0.6235
# Epoch: 60/100, Loss: 0.6025
# Epoch: 70/100, Loss: 0.5829
# Epoch: 80/100, Loss: 0.5645
# Epoch: 90/100, Loss: 0.5472
# Epoch: 100/100, Loss: 0.5310
# Training finished.

This example demonstrates a foundational neural network. In real-world applications, you’d use larger datasets, potentially more complex architectures, and techniques like data normalization, validation sets, and more sophisticated hyperparameter tuning. For further learning, explore PyTorch’s documentation on torch.nn, torch.optim, and data handling with Dataset and DataLoader.