TensorFlow - Linear Regression

TensorFlow - Linear Regression

Linear regression is a fundamental supervised machine learning algorithm used to model the relationship between a dependent variable (target) and one or more independent variables (features) by fitting a linear equation to the observed data. This chapter demonstrates how to implement a simple linear regression model using TensorFlow and its Keras API.

If ‘y’ is the dependent variable and ‘x’ is the independent variable, the linear regression equation for a single feature is:

Y = W*x + b

Where ‘W’ is the weight (slope) and ‘b’ is the bias (intercept). The goal of training is to find the optimal values for ‘W’ and ‘b’ that minimize the difference between predicted ‘Y’ values and actual ‘Y’ values. This is typically done by minimizing a cost function using an optimization algorithm.

Key concepts involved:

• Cost Function: Measures the error between predicted and actual values (e.g., Mean Squared Error - MSE).
• Gradient Descent: An optimization algorithm used to minimize the cost function by iteratively adjusting weights and biases.

A simple representation of linear regression involves finding the best-fitting straight line through a scatter plot of data points.

If we have data points (x, y), linear regression aims to find a line Y = Wx + b that best represents these points.

Steps to implement Linear Regression with TensorFlow Keras

We’ll generate some synthetic data and then build a model to learn the underlying linear relationship.

Step 1: Import necessary libraries and generate data

We’ll use NumPy for numerical operations and Matplotlib for plotting.

import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt

# For reproducibility
np.random.seed(42)
tf.random.set_seed(42)

# Generate synthetic data
# True parameters
TRUE_W = 2.5
TRUE_b = 0.8
NUM_EXAMPLES = 500

X_data = np.random.rand(NUM_EXAMPLES) # Input feature (x values between 0 and 1)
noise = np.random.randn(NUM_EXAMPLES) * 0.2 # Some noise
y_data = TRUE_W * X_data + TRUE_b + noise   # Target variable

# Reshape X_data to be (num_examples, 1) for Keras model input
X_data = X_data.reshape(-1, 1)

Step 2: Visualize the generated data

Plotting the data helps understand the relationship we’re trying to model.

plt.figure(figsize=(8, 6))
plt.scatter(X_data, y_data, label='Original data', alpha=0.6)
plt.plot(X_data, TRUE_W * X_data + TRUE_b, label='True underlying relationship', color='red', linewidth=2)
plt.xlabel('X')
plt.ylabel('y')
plt.title('Synthetic Data for Linear Regression')
plt.legend()
plt.grid(True)
plt.show()

The plot will show scattered points around a true linear relationship.

Step 3: Define the Linear Regression model using Keras

A linear regression model with one input feature and one output can be represented by a single Dense layer with one unit.

model = keras.Sequential([
    layers.Dense(units=1, input_shape=[1]) # One unit, one input feature
])

model.summary()

The Dense layer implements the operation: output = activation(dot(input, kernel) + bias). For linear regression, the default linear activation is used. kernel corresponds to ‘W’ and bias to ‘b’.

Step 4: Compile the model

Specify the optimizer and loss function.

model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.1),
              loss='mean_squared_error') # MSE is common for regression

We use Stochastic Gradient Descent (SGD) as the optimizer and Mean Squared Error (MSE) as the loss function.

Step 5: Train the model

Fit the model to our synthetic data.

print("\nTraining the model...")
epochs = 100
history = model.fit(X_data, y_data, epochs=epochs, verbose=0) # verbose=0 to keep output clean

# Retrieve the learned weights
learned_W = model.layers[0].get_weights()[0][0][0]
learned_b = model.layers[0].get_weights()[1][0]

print(f"Epochs run: {epochs}")
print(f"Learned W: {learned_W:.4f}, Learned b: {learned_b:.4f}")
print(f"True W: {TRUE_W}, True b: {TRUE_b}")

After training, the learned weights ‘W’ and ‘b’ should be close to our TRUE_W and TRUE_b.

Step 6: Visualize the results

Plot the original data along with the learned regression line.

plt.figure(figsize=(8, 6))
plt.scatter(X_data, y_data, label='Original data', alpha=0.6)
plt.plot(X_data, TRUE_W * X_data + TRUE_b, label='True relationship', color='red', linewidth=2, linestyle='--')
plt.plot(X_data, model.predict(X_data), label='Learned regression line', color='green', linewidth=2)
plt.xlabel('X')
plt.ylabel('y')
plt.title('Linear Regression Fit')
plt.legend()
plt.grid(True)
plt.show()

The complete code will generate data, build a Keras model, train it, and then plot the learned line against the original data points, showing how TensorFlow can learn the linear relationship. This demonstrates a modern, concise way to perform linear regression using TensorFlow 2.x and Keras.

Expected output from training (values may vary slightly):

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 dense (Dense)               (None, 1)                 2

=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________

Training the model...
Epochs run: 100
Learned W: 2.4878, Learned b: 0.8111
True W: 2.5, True b: 0.8