Keras - Convolution Neural Network

Keras - Convolutional Neural Networks (CNN)

Convolutional Neural Networks (CNNs or ConvNets) are a class of deep neural networks particularly well-suited for processing grid-like data, most notably images. They leverage the principles of shared weights (using convolutional filters) and spatial hierarchies to automatically learn features from the input data, making them highly effective for tasks like image classification, object detection, and image segmentation.

Let’s adapt the MNIST handwritten digit classification problem, previously solved with an MLP, to use a CNN built with tf.keras. CNNs typically outperform MLPs on image tasks because they explicitly model spatial relationships.

Conceptual CNN Architecture for MNIST:

Input Image (28x28x1) -> Conv2D Layer (feature extraction) -> MaxPooling2D (downsampling) -> Conv2D Layer -> MaxPooling2D -> Flatten Layer -> Dense Layer (classification head) -> Output Layer (softmax).

Model Specifications:

• Dataset: MNIST (handwritten digits).
• Preprocessing: Reshape images to include channel dimension (e.g., 28x28x1), normalize pixel values.
• Input Layer: Implicitly defined by the first Conv2D layer’s input_shape.
• Convolutional Layer 1: Conv2D with 32 filters, kernel size (3,3), ReLU activation.
• Convolutional Layer 2: Conv2D with 64 filters, kernel size (3,3), ReLU activation.
• Pooling Layer: MaxPooling2D with pool size (2,2) after the second Conv layer.
• Regularization: Dropout after pooling.
• Flatten Layer: Flatten to convert 2D feature maps into a 1D vector for the classifier.
• Dense Classifier Head: Dense layer (e.g., 128 units, ReLU activation), followed by Dropout.
• Output Layer: Dense layer with 10 units (one for each digit) and softmax activation.
• Compilation: categorical_crossentropy or sparse_categorical_crossentropy loss, Adam optimizer (or Adadelta as in original example), accuracy metric.

Step 1: Import Necessary Modules

import tensorflow as tf
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras import backend as K # To check image data format
import numpy as np

Step 2: Define Parameters and Load Data

batch_size = 128
num_classes = 10
epochs = 12 # Number of training epochs

# Input image dimensions
img_rows, img_cols = 28, 28

# Load the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

Step 3: Preprocess the Data

CNNs require input data with a channel dimension. We also normalize pixel values to the [0, 1] range.

# Determine input shape based on Keras backend data format ('channels_first' or 'channels_last')
if K.image_data_format() == 'channels_first':
    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else: # 'channels_last'
    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

# Convert data type to float32 and normalize
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# Convert class vectors to binary class matrices (one-hot encoding)
# Necessary for 'categorical_crossentropy' loss
y_train_cat = tf.keras.utils.to_categorical(y_train, num_classes)
y_test_cat = tf.keras.utils.to_categorical(y_test, num_classes)

# If using 'sparse_categorical_crossentropy', keep y_train/y_test as integers.

Step 4: Build the CNN Model

model = Sequential([
    tf.keras.Input(shape=input_shape), # Define input shape
    Conv2D(32, kernel_size=(3, 3), activation='relu'),
    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2, 2)),
    Dropout(0.25),
    Flatten(), # Flatten the 2D feature maps to 1D
    Dense(128, activation='relu'),
    Dropout(0.5),
    Dense(num_classes, activation='softmax')
], name="mnist_cnn")

model.summary()

Step 5: Compile the Model

Configure the learning process. We’ll use categorical cross-entropy because labels are one-hot encoded.

# Optimizer: Adadelta() was used in the original example, Adam is also common.
optimizer = tf.keras.optimizers.Adam() # Or tf.keras.optimizers.Adadelta()

model.compile(loss=tf.keras.losses.categorical_crossentropy,
              optimizer=optimizer,
              metrics=['accuracy'])

Step 6: Train the Model

Fit the model to the training data.

print("\nTraining CNN Model...")
history = model.fit(x_train, y_train_cat, # Use one-hot encoded labels
                    batch_size=batch_size,
                    epochs=epochs,
                    verbose=1,
                    validation_data=(x_test, y_test_cat))

print("\nTraining Complete.")

Monitor the training and validation accuracy/loss printed during training. CNNs often achieve higher accuracy on MNIST compared to simple MLPs.

0.9820

# Example Output Snippet (values depend on optimizer, epochs etc.):
# Epoch 1/12
# Epoch 2/12
# 469/469 [==============================] - 8s 18ms/step - loss: 0.0876 - accuracy: 0.9741 - val_loss: 0.0396 - val_accuracy: 0.9869
# ...
# Epoch 12/12
# 469/469 [==============================] - 9s 18ms/step - loss: 0.0186 - accuracy: 0.9941 - val_loss: 0.0272 - val_accuracy: 0.9917

Step 7: Evaluate the Model

Assess the final performance on the test set.

print("\nEvaluating Model...")
score = model.evaluate(x_test, y_test_cat, verbose=0)
print(f'Test loss: {score[0]:.4f}')
print(f'Test accuracy: {score[1]:.4f}')

CNNs often achieve >99% accuracy on MNIST relatively easily.

Step 8: Make Predictions (Optional)

Use the trained model to predict labels for test images.

# Get predictions (probabilities)
predictions_probabilities = model.predict(x_test)

# Get predicted labels (index of max probability)
predicted_labels = np.argmax(predictions_probabilities, axis=1)

# Compare first few predictions with actual labels
print("\nFirst 5 Predicted labels:", predicted_labels[:5])
print("First 5 Actual labels:   ", y_test[:5]) # Original integer labels

This example demonstrates the typical workflow for building, training, and evaluating a CNN for image classification using tf.keras, highlighting the use of Conv2D and MaxPooling2D layers.