AI with Python – Computer Vision

AI with Python – Computer Vision

Computer Vision is a field of Artificial Intelligence that enables computers and systems to derive meaningful information from digital images, videos, and other visual inputs — and take actions or make recommendations based on that information. It seeks to model and replicate human vision using computer software and hardware.

Understanding Computer Vision

Computer Vision aims to reconstruct, interpret, and understand a 3D scene or its components from 2D images or sequences of images. It involves acquiring, processing, analyzing, and understanding digital images to extract high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions.

Computer Vision Hierarchy (Levels of Processing)

Computer vision tasks can be broadly categorized into levels of complexity:

• Low-level Vision: Involves basic image processing operations like noise reduction, contrast enhancement, and feature extraction (e.g., edges, corners, textures).
• Mid-level Vision: Focuses on tasks like image segmentation (partitioning an image into multiple segments), object recognition (identifying objects), and motion tracking.
• High-level Vision: Deals with scene understanding, activity recognition, and interpreting the intentions or behaviors depicted in a visual scene.

Computer Vision vs. Image Processing

Image Processing typically involves image-to-image transformations. The input is an image, and the output is also an image (e.g., enhancing an image, removing noise).

Computer Vision, while often using image processing techniques as a first step, goes further. Its goal is to produce an explicit, meaningful description or interpretation of the physical objects and scenes from images or videos. The output is often symbolic information, a decision, or a model of the scene.

Applications of Computer Vision

Computer vision has a wide array of applications across various industries:

Robotics

• Localization & Navigation: Helping robots determine their location and navigate autonomously (e.g., SLAM - Simultaneous Localization and Mapping).
• Obstacle Avoidance: Detecting and avoiding obstacles in the environment.
• Assembly & Manipulation: Guiding robotic arms for tasks like welding, painting, or picking and placing objects.
• Human-Robot Interaction (HRI): Enabling robots to perceive and interact with humans more naturally.

Healthcare & Medicine

• Medical Image Analysis: Classification and detection of anomalies (e.g., tumors in X-rays, CT scans, MRIs).
• 2D/3D Segmentation: Outlining organs or tissues in medical images.
• 3D Organ Reconstruction: Creating 3D models from scans like MRI or ultrasound.
• Surgical Assistance: Vision-guided robotic surgery.

Security & Surveillance

• Biometrics: Face recognition, iris scanning, fingerprint identification.
• Surveillance: Detecting suspicious activities, unauthorized access, or monitoring crowds.

Transportation

• Autonomous Vehicles: Self-driving cars use computer vision for perception, lane detection, traffic sign recognition, and pedestrian detection.
• Driver Assistance Systems (ADAS): Features like lane departure warning, adaptive cruise control.
• Traffic Monitoring: Analyzing traffic flow and detecting incidents.

Industrial Automation

• Quality Control: Automated inspection for defect detection in manufacturing.
• Assembly Line Automation: Guiding robots and verifying assembly steps.
• Barcode & Package Label Reading: Automated sorting and tracking.
• Document Analysis (OCR): Optical Character Recognition for digitizing text.

Installing Useful Packages for Computer Vision in Python

The primary library for computer vision tasks in Python is OpenCV (Open Source Computer Vision Library). It provides a vast collection of algorithms and functions for real-time computer vision.

OpenCV

OpenCV is written in C++ and has interfaces for Python, Java, and MATLAB. To install the Python bindings for OpenCV, use pip:

pip install opencv-python

This package includes the main modules of OpenCV. For additional modules (e.g., some patented algorithms), you might need opencv-contrib-python:

pip install opencv-contrib-python

If you are using Anaconda, you can install OpenCV from the conda-forge channel:

conda install -c conda-forge opencv

We will also use NumPy for array manipulations, which is usually installed as a dependency of OpenCV, and Matplotlib for displaying images in some contexts (though OpenCV has its own display capabilities).

Basic Image Operations with OpenCV

Most computer vision applications involve reading images as input, processing them, and sometimes producing images as output. Let’s cover fundamental operations.

Reading, Displaying, and Writing Images

OpenCV provides functions for these tasks:

• cv2.imread(filepath, flags): Reads an image from a file. The flags argument specifies the color mode (e.g., cv2.IMREAD_COLOR, cv2.IMREAD_GRAYSCALE).
• cv2.imshow(window_name, image): Displays an image in a window. Requires cv2.waitKey() and cv2.destroyAllWindows() for proper handling.
• cv2.imwrite(filepath, image): Saves an image to a file.

Example: Load, Display, and Save an Image

First, ensure you have an image file (e.g., my_image.jpg) in your working directory or provide the full path.

import cv2
import numpy as np

# Define image path (replace with your image file)
image_path = 'my_image.jpg' # Make sure this image exists

# Try to load the image
# As a fallback, create a dummy image if the specified one is not found
img = cv2.imread(image_path, cv2.IMREAD_COLOR)

if img is None:
    print(f"Error: Could not read image at {image_path}. Using a dummy image instead.")
    # Create a dummy 300x400 color image (BGR format)
    img = np.zeros((300, 400, 3), dtype=np.uint8)
    cv2.putText(img, 'Dummy Image', (50, 150), cv2.FONT_HERSHEY_SIMPLEX,
                1.5, (0, 255, 0), 2, cv2.LINE_AA)

# Display the image
cv2.imshow('Original Image', img)

print("Press any key to close the image window...")
# waitKey(0) waits indefinitely for a key press
cv2.waitKey(0)

# Save the image in a different format (e.g., PNG)
output_path = 'output_image.png'
success = cv2.imwrite(output_path, img)
if success:
    print(f"Image successfully saved to {output_path}")
else:
    print(f"Error: Failed to save image to {output_path}")

# Destroy all OpenCV windows
cv2.destroyAllWindows()

When you run this code, an image (either my_image.jpg or a dummy green image with text) will be displayed in a window titled ‘Original Image’. Pressing any key will close the window, and the script will attempt to save the image as output_image.png.

Color Space Conversion

OpenCV reads images in BGR (Blue, Green, Red) order by default, unlike the more common RGB order used in libraries like Matplotlib. It’s often necessary to convert images between color spaces (e.g., BGR to Grayscale, BGR to HSV). The cv2.cvtColor(input_image, flag) function is used for this.

Example: Convert BGR to Grayscale

import cv2
import numpy as np

image_path = 'my_image.jpg' # Use the same image or a new one
img_bgr = cv2.imread(image_path, cv2.IMREAD_COLOR)

if img_bgr is None:
    print(f"Error: Could not read image at {image_path}. Using a dummy BGR image.")
    img_bgr = np.random.randint(0, 256, (300, 400, 3), dtype=np.uint8)

# Convert BGR image to Grayscale
img_gray = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2GRAY)

# Display both images
cv2.imshow('BGR Image', img_bgr)
cv2.imshow('Grayscale Image', img_gray)

print("Press any key to close image windows...")
cv2.waitKey(0)
cv2.destroyAllWindows()

This script will display two windows: one with the original color image (in BGR format as loaded by OpenCV) and another with its grayscale version. Grayscale images are single-channel and are often used to simplify processing or for algorithms that don’t require color information.

Edge Detection

Edges are significant local changes in image intensity and represent important structural information. Edge detection is a fundamental step in many computer vision applications. The Canny edge detector is a popular and effective algorithm.

Example: Canny Edge Detection

import cv2
import numpy as np

image_path = 'my_image.jpg'
img_color = cv2.imread(image_path, cv2.IMREAD_COLOR)

if img_color is None:
    print(f"Error: Could not read image at {image_path}. Using a dummy image.")
    # Create a dummy image with some shapes for edge detection
    img_color = np.zeros((300, 400, 3), dtype=np.uint8)
    cv2.rectangle(img_color, (50, 50), (150, 150), (0, 255, 0), -1) # Green square
    cv2.circle(img_color, (250, 100), 50, (0, 0, 255), -1)    # Red circle

# Convert to grayscale first, as Canny works on single-channel images
img_gray_for_canny = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY)

# Apply Canny edge detection
# The two thresholds (100 and 200) are for hysteresis thresholding.
# Adjust these thresholds based on your image content.
edges = cv2.Canny(img_gray_for_canny, threshold1=100, threshold2=200)

# Display the original and edge-detected images
cv2.imshow('Original Image for Edges', img_color)
cv2.imshow('Canny Edges', edges)

print("Press any key to close image windows...")
cv2.waitKey(0)
cv2.destroyAllWindows()

# Optionally save the edges image
cv2.imwrite('edges_output.jpg', edges)

This program will show the original image and another image highlighting the detected edges in white on a black background. The Canny algorithm involves noise reduction, gradient calculation, non-maximum suppression, and hysteresis thresholding to produce clean edges.

Face Detection using Haar Cascades

Face detection is a classic computer vision task. OpenCV provides pre-trained Haar cascade classifiers for detecting objects like faces, eyes, and smiles. These classifiers are based on machine learning and use Haar-like features.

Loading the Haar Cascade Classifier

OpenCV includes XML files for various pre-trained Haar cascades. You need to provide the path to the appropriate XML file (e.g., haarcascade_frontalface_default.xml).

import cv2
import numpy as np

# Load the pre-trained Haar Cascade for frontal face detection
# Use cv2.data.haarcascades to get the path to OpenCV's built-in cascades
cascade_path = cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'
face_cascade = cv2.CascadeClassifier(cascade_path)

if face_cascade.empty():
    raise IOError('Unable to load the face cascade classifier xml file')

# Load an image with faces (replace 'image_with_faces.jpg')
image_path_faces = 'image_with_faces.jpg'
img_faces = cv2.imread(image_path_faces)

if img_faces is None:
    print(f"Error: Could not read image at {image_path_faces}. Creating a dummy image.")
    # Create a dummy image (you'd ideally use an image with actual faces for testing)
    img_faces = np.zeros((480, 640, 3), dtype=np.uint8)
    # Draw a few dummy "faces" (simple rectangles)
    cv2.rectangle(img_faces, (100, 100), (200, 220), (200,200,200), -1) # A gray box
    cv2.rectangle(img_faces, (300, 150), (450, 300), (200,200,200), -1) # Another gray box

# Convert to grayscale (face detection is typically done on grayscale images)
img_gray_faces = cv2.cvtColor(img_faces, cv2.COLOR_BGR2GRAY)

# Detect faces in the image
# detectMultiScale(image, scaleFactor, minNeighbors)
# scaleFactor: How much the image size is reduced at each image scale.
# minNeighbors: How many neighbors each candidate rectangle should have to retain it.
faces_detected = face_cascade.detectMultiScale(img_gray_faces, scaleFactor=1.1, minNeighbors=5)

print(f"Found {len(faces_detected)} faces!")

# Draw rectangles around the detected faces
for (x, y, w, h) in faces_detected:
    cv2.rectangle(img_faces, (x, y), (x + w, y + h), (0, 255, 0), 3) # Green rectangle

# Display the image with detected faces
cv2.imshow('Faces Detected', img_faces)

print("Press any key to close the image window...")
cv2.waitKey(0)
cv2.destroyAllWindows()

cv2.imwrite('faces_detected_output.jpg', img_faces)

When you run this, if faces are present in image_with_faces.jpg (or the dummy image), they will be highlighted with green rectangles. The scaleFactor and minNeighbors parameters might need tuning for optimal performance on different images.

Eye Detection using Haar Cascades

Similar to face detection, eye detection can also be performed using Haar cascades. This is often done as a secondary step after face detection to locate eyes within the detected face regions, improving accuracy.

import cv2
import numpy as np

face_cascade_path = cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'
eye_cascade_path = cv2.data.haarcascades + 'haarcascade_eye.xml'
# For eyes with glasses, 'haarcascade_eye_tree_eyeglasses.xml' might be better

face_cascade_eyes = cv2.CascadeClassifier(face_cascade_path)
eye_cascade = cv2.CascadeClassifier(eye_cascade_path)

if face_cascade_eyes.empty() or eye_cascade.empty():
    raise IOError('Unable to load Haar cascade xml files')

image_path_eyes = 'image_with_faces.jpg' # Use an image with clear faces and eyes
img_for_eyes = cv2.imread(image_path_eyes)

if img_for_eyes is None:
    print(f"Error: Could not read {image_path_eyes}. Using dummy.")
    img_for_eyes = np.zeros((480, 640, 3), dtype=np.uint8)
    # Dummy face region
    cv2.rectangle(img_for_eyes, (100,100), (300,350), (200,200,200), -1)
    # Dummy eye regions within the face
    cv2.circle(img_for_eyes, (160,180), 15, (50,50,50), -1)
    cv2.circle(img_for_eyes, (240,180), 15, (50,50,50), -1)

img_gray_for_eyes = cv2.cvtColor(img_for_eyes, cv2.COLOR_BGR2GRAY)

# First, detect faces
faces = face_cascade_eyes.detectMultiScale(img_gray_for_eyes, 1.1, 5)

# Then, for each detected face, detect eyes
for (fx, fy, fw, fh) in faces:
    cv2.rectangle(img_for_eyes, (fx, fy), (fx + fw, fy + fh), (255, 0, 0), 2) # Blue for face
    roi_gray = img_gray_for_eyes[fy:fy+fh, fx:fx+fw] # Region of Interest (face) in grayscale
    roi_color = img_for_eyes[fy:fy+fh, fx:fx+fw]    # Region of Interest (face) in color

    eyes = eye_cascade.detectMultiScale(roi_gray, scaleFactor=1.1, minNeighbors=5)
    for (ex, ey, ew, eh) in eyes:
        cv2.rectangle(roi_color, (ex, ey), (ex + ew, ey + eh), (0, 255, 0), 2) # Green for eyes

cv2.imshow('Eyes Detected', img_for_eyes)
print("Press any key to close...")
cv2.waitKey(0)
cv2.destroyAllWindows()

cv2.imwrite('eyes_detected_output.jpg', img_for_eyes)

This script first detects faces (blue rectangles) and then searches for eyes (green rectangles) only within those detected face regions. This hierarchical approach is generally more robust.

Modern Computer Vision: Deep Learning Approaches

While classical methods like Haar Cascades are useful, modern computer vision heavily relies on Deep Learning, particularly Convolutional Neural Networks (CNNs). CNNs have achieved state-of-the-art performance in tasks like:

• Image Classification: Assigning a label to an entire image (e.g., ‘cat’, ‘dog’, ‘car’).
• Object Detection: Identifying and locating multiple objects within an image (e.g., YOLO, SSD, Faster R-CNN).
• Image Segmentation: Classifying each pixel in an image (semantic segmentation) or distinguishing object instances (instance segmentation).
• Facial Recognition: Advanced face identification and verification systems.
• Image Generation: Creating new images (e.g., Generative Adversarial Networks - GANs).

Libraries like TensorFlow, Keras, and PyTorch are used to build and train these deep learning models. OpenCV also includes a DNN (Deep Neural Network) module that can run pre-trained deep learning models for inference, allowing you to leverage powerful models without needing to train them from scratch.

This introduction provides a starting point. The field of computer vision is vast and rapidly evolving, with deep learning opening up new frontiers.