TensorFlow - XOR Implementation

The XOR (exclusive OR) problem is a classic example in neural networks because it’s a linearly inseparable problem. This means a simple linear classifier cannot solve it. A neural network with at least one hidden layer is required. This chapter demonstrates how to implement a neural network to solve the XOR problem using TensorFlow and its Keras API.

Input A	Input B	Output (A XOR B)
0	0	0
0	1	1
1	0	1
1	1	0

The XOR function outputs 1 if the inputs are different, and 0 if they are the same. We will build a small neural network to learn this mapping.

We’ll use tf.keras to define, compile, and train the model.

# Import necessary modules
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np

# For reproducibility
np.random.seed(42)
tf.random.set_seed(42)

# Define the XOR input data (features) and output data (labels)
X_train = np.array([[0, 0],
                    [0, 1],
                    [1, 0],
                    [1, 1]], dtype="float32")

y_train = np.array([[0],
                    [1],
                    [1],
                    [0]], dtype="float32")

# Define the neural network model using Keras Sequential API
model = keras.Sequential([
    # Input layer: Implicitly defined by input_shape in the first layer
    # Hidden layer: 2 units, 'sigmoid' activation often used for XOR
    # A more common modern choice for hidden layers is 'relu', but 'sigmoid' also works here.
    layers.Dense(units=8, activation='relu', input_shape=(2,)), # 2 input features
    # Output layer: 1 unit (for binary output), 'sigmoid' activation for probability
    layers.Dense(units=1, activation='sigmoid')
])

# Compile the model
# Optimizer: Adam is a good general-purpose optimizer
# Loss: Binary Crossentropy is suitable for binary classification (0 or 1)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Print model summary
model.summary()

# Train the model
print("\nTraining the model...")
# epochs: Number of times to iterate over the entire dataset
# verbose=0 to keep output clean for the tutorial, set to 1 or 2 for more details
history = model.fit(X_train, y_train, epochs=100, verbose=0)

# Evaluate the model (optional, as the dataset is very small)
loss, accuracy = model.evaluate(X_train, y_train, verbose=0)
print(f"\nTraining Loss: {loss:.4f}")
print(f"Training Accuracy: {accuracy:.4f}")

# Make predictions
print("\nPredictions (Probabilities):")
predictions_prob = model.predict(X_train)
print(predictions_prob)

# Convert probabilities to binary predictions (0 or 1)
predictions_binary = (predictions_prob > 0.5).astype(int)
print("\nBinary Predictions (0 or 1):")
print(predictions_binary)

# Display input, true output, and predicted output
print("\nInput | True Output | Predicted Output (Prob) | Predicted Output (Binary)")
print("------------------------------------------------------------------------")
for i in range(len(X_train)):
    print(f"{X_train[i]} |     {y_train[i][0]}     |        {predictions_prob[i][0]:.4f}       |           {predictions_binary[i][0]}")

This code defines a small neural network, compiles it with an Adam optimizer and binary cross-entropy loss, trains it on the XOR data, and then prints the predictions. The hidden layer with non-linear activation (like ‘relu’ or ‘sigmoid’) is key to solving the XOR problem.

Expected output (will vary slightly due to random initializations, especially with few epochs or different learning rates. More epochs or different hyperparameters might be needed for perfect XOR):

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 dense (Dense)               (None, 8)                 24

 dense_1 (Dense)             (None, 1)                 9

=================================================================
Total params: 33
Trainable params: 33
Non-trainable params: 0
_________________________________________________________________

Training the model...

Training Loss: 0.0128
Training Accuracy: 1.0000

Predictions (Probabilities):
1/1 [==============================] - 0s 21ms/step
[[0.00621046]
 [0.99280304]
 [0.9930576 ]
 [0.01085009]]

Binary Predictions (0 or 1):
[[0]
 [1]
 [1]
 [0]]

Input | True Output | Predicted Output (Prob) | Predicted Output (Binary)
------------------------------------------------------------------------
[0. 0.] |     0.0     |        0.0062       |           0
[0. 1.] |     1.0     |        0.9928       |           1
[1. 0.] |     1.0     |        0.9931       |           1
[1. 1.] |     0.0     |        0.0109       |           0