Keras - Deep learning

Keras - Deep Learning with tf.keras

Keras is a high-level API for building and training neural networks. It focuses on enabling fast experimentation and user-friendliness. Originally a standalone library capable of running on multiple backends (TensorFlow, Theano, CNTK), Keras is now the official high-level API integrated directly within TensorFlow (tf.keras). This integration provides a seamless workflow within the TensorFlow ecosystem.

tf.keras offers a complete framework for creating nearly any type of neural network, from simple sequential models to complex architectures with multiple inputs/outputs or custom components. Let’s explore the core components of the tf.keras API.

Core Components of tf.keras

The tf.keras API can be broadly divided into three main categories:

• Models: The main container that groups layers together and provides methods for training, evaluation, and prediction (tf.keras.Model).
• Layers: The fundamental building blocks of a neural network, performing computations and transformations on data (tf.keras.layers).
• Core Modules: Supporting components like activation functions, optimizers, loss functions, metrics, initializers, regularizers, and utilities (tf.keras.activations, tf.keras.optimizers, tf.keras.losses, tf.keras.metrics, etc.).

Conceptual Structure: You define a Model by assembling Layers. Layers utilize functionalities from Core Modules (like activation functions or initializers). The Model is then compiled using components from Core Modules (Optimizer, Loss, Metrics) and trained on data.

Models (tf.keras.Model)

tf.keras provides two primary ways to define models:

1. Sequential API (tf.keras.Sequential)

The Sequential model is suitable for simple stacks of layers where each layer has exactly one input tensor and one output tensor. It’s intuitive and easy to use for many common network architectures.

Example:

import tensorflow as tf

# Define a Sequential model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(512, activation='relu', input_shape=(784,)), # Input layer requires input_shape
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax') # Output layer
])

# Alternatively, add layers incrementally
# model = tf.keras.Sequential()
# model.add(tf.keras.layers.Dense(512, activation='relu', input_shape=(784,)))
# model.add(tf.keras.layers.Dropout(0.2))
# model.add(tf.keras.layers.Dense(10, activation='softmax'))

Here, we create a model with a Dense input layer, a Dropout layer for regularization, and a Dense output layer with softmax activation (suitable for multi-class classification).

2. Functional API

The Functional API provides more flexibility for creating complex models. It allows for models with non-linear topology, shared layers, and multiple inputs or outputs. You define layers as callable objects and connect them explicitly.

Example (Conceptual):

import tensorflow as tf

# Define input layer
inputs = tf.keras.Input(shape=(784,))

# Define hidden layers, connecting them
x = tf.keras.layers.Dense(64, activation='relu')(inputs)
x = tf.keras.layers.Dense(64, activation='relu')(x)

# Define output layer
outputs = tf.keras.layers.Dense(10, activation='softmax')(x)

# Create the Model
model = tf.keras.Model(inputs=inputs, outputs=outputs)

This creates the same basic structure as the Sequential example but offers the potential to build much more intricate graphs.

3. Model Subclassing

For maximum flexibility and control, you can create custom models by subclassing tf.keras.Model. This is useful for research or implementing highly customized architectures. You define layers in __init__ and implement the forward pass logic in the call method.

Layers (tf.keras.layers)

Layers are the core building blocks. Each layer encapsulates specific computations and transformations. tf.keras.layers provides a wide variety of pre-built layers:

• Core Layers: Dense (fully connected), Activation, Dropout, Flatten, Reshape, InputLayer, etc.
• Convolutional Layers: Conv1D, Conv2D, Conv3D, SeparableConv2D, DepthwiseConv2D, etc. (for CNNs).
• Pooling Layers: MaxPooling1D, MaxPooling2D, AveragePooling2D, GlobalAveragePooling2D, etc. (for CNNs).
• Recurrent Layers: LSTM, GRU, SimpleRNN, RNN, Bidirectional wrapper (for RNNs).
• Normalization Layers: BatchNormalization, LayerNormalization.
• Preprocessing Layers: Resizing, Rescaling, CenterCrop, Normalization, TextVectorization, etc. (Often used as the first layers in a model or within tf.data pipelines).
• Attention Layers: Attention, AdditiveAttention, MultiHeadAttention.

You combine these layers using the Sequential or Functional API to build your desired network architecture. Keras also allows creating custom layers by subclassing tf.keras.layers.Layer.

Core Modules

These modules provide essential functions and classes used during model definition, compilation, and training:

• tf.keras.activations: Activation functions (e.g., relu, sigmoid, softmax, tanh, selu). These introduce non-linearity.
• tf.keras.losses: Loss functions (e.g., BinaryCrossentropy, CategoricalCrossentropy, SparseCategoricalCrossentropy, MeanSquaredError, MeanAbsoluteError). Quantify the model’s error.
• tf.keras.optimizers: Optimization algorithms (e.g., Adam, SGD, RMSprop, Adagrad). Update model weights to minimize loss.
• tf.keras.metrics: Evaluation metrics (e.g., Accuracy, Precision, Recall, AUC, MeanAbsoluteError). Assess model performance.
• tf.keras.initializers: Weight initialization schemes (e.g., GlorotUniform, HeNormal, Zeros, Ones). Set initial weights for layers.
• tf.keras.regularizers: Regularization techniques (e.g., L1, L2, L1L2). Apply penalties to layer parameters or activations to prevent overfitting.
• tf.keras.callbacks: Objects passed to model.fit() to perform actions at various stages of training (e.g., ModelCheckpoint to save the model, EarlyStopping to stop training early, TensorBoard for visualization).
• tf.keras.utils: Utility functions (e.g., to_categorical for one-hot encoding, plot_model for visualizing architecture, load_img, img_to_array for image loading).

By combining these components, tf.keras provides a powerful yet user-friendly environment for developing and deploying deep learning models within the TensorFlow ecosystem.