NumPy - Statistical Functions

NumPy - Statistical Functions

NumPy provides a suite of functions for performing common statistical calculations on array data. These functions can operate on the entire array or along a specified axis for multi-dimensional arrays.

A crucial parameter for many statistical functions when working with multi-dimensional arrays is axis. It specifies the axis along which the computation should be performed:

• axis=0: Operates along the rows (collapses rows, result for each column).
• axis=1: Operates along the columns (collapses columns, result for each row).
• axis=None (default): Operates on the flattened array (all elements).

Many statistical functions are also available as methods of the ndarray object itself (e.g., a.min(), a.mean(axis=0)).

Minimum and Maximum (numpy.amin, numpy.amax, numpy.ptp)

• numpy.amin() or ndarray.min(): Finds the minimum value.
• numpy.amax() or ndarray.max(): Finds the maximum value.
• numpy.ptp(): Calculates the range (Peak-to-Peak = max - min).

Example: Min, Max, and Range

import numpy as np

a = np.array([[3, 7, 5],
              [8, 4, 3],
              [2, 4, 9]])

print(f"Our array:\n{a}\n")

# Overall min/max/range
print(f"Overall minimum (np.amin): {np.amin(a)}")
print(f"Overall maximum (a.max()): {a.max()}")
print(f"Overall range (np.ptp): {np.ptp(a)}\n")

# Operations along axes
print(f"Minimum along axis 0 (columns): {np.amin(a, axis=0)}")
print(f"Maximum along axis 1 (rows): {np.amax(a, axis=1)}")
print(f"Range along axis 0 (columns): {np.ptp(a, axis=0)}")
print(f"Range along axis 1 (rows): {np.ptp(a, axis=1)}")

Output:

Our array:
[[3 7 5]
 [8 4 3]
 [2 4 9]]

Overall minimum (np.amin): 2
Overall maximum (a.max()): 9
Overall range (np.ptp): 7

Minimum along axis 0 (columns): [2 4 3]
Maximum along axis 1 (rows): [7 8 9]
Range along axis 0 (columns): [6 3 6]
Range along axis 1 (rows): [4 5 7]

Percentiles and Median (numpy.percentile, numpy.median)

• numpy.percentile(a, q, axis=None, ...): Computes the q-th percentile(s) of the data. q must be between 0 and 100.
• numpy.median(a, axis=None, ...): Computes the median (50th percentile).

Percentiles indicate the value below which a given percentage of observations fall. The median separates the lower half from the higher half of the data.

Example: Percentiles and Median

import numpy as np

a = np.array([[30, 40, 70],
              [80, 20, 10],
              [50, 90, 60]])

print(f"Our array:\n{a}\n")

# Median (50th percentile) overall
print(f"Overall median: {np.median(a)}")
# Also: np.percentile(a, 50)

# 25th percentile (first quartile) along columns
print(f"25th percentile (axis=0): {np.percentile(a, 25, axis=0)}")

# Median along rows
print(f"Median (axis=1): {np.median(a, axis=1)}")

Output:

Our array:
[[30 40 70]
 [80 20 10]
 [50 90 60]]

Overall median: 50.0
25th percentile (axis=0): [40. 30. 35.]
Median (axis=1): [40. 20. 60.]

Mean and Average (numpy.mean, numpy.average)

• numpy.mean() or ndarray.mean(): Computes the arithmetic mean (average).
• numpy.average(): Computes the weighted average. If weights are not provided, it’s equivalent to mean.

Example: Mean and Weighted Average

import numpy as np

a = np.array([[1, 2, 3],
              [4, 5, 6]])
print(f"Our array (a):\n{a}\n")

# Calculate the mean
print(f"Overall mean: {np.mean(a)}")
print(f"Mean along axis 0 (columns): {a.mean(axis=0)}")
print(f"Mean along axis 1 (rows): {a.mean(axis=1)}\n")

# Calculate weighted average along rows
weights = np.array([1, 2, 3]) # Weights for columns
# For row 0: (1*1 + 2*2 + 3*3) / (1+2+3) = 14 / 6 = 2.333
# For row 1: (1*4 + 2*5 + 3*6) / (1+2+3) = 32 / 6 = 5.333
print(f"Weights: {weights}")
print(f"Weighted average (axis=1): {np.average(a, axis=1, weights=weights)}")

# Average can also return sum of weights
avg, sum_weights = np.average(a, axis=1, weights=weights, returned=True)
print(f"Weighted avg (returned=True): {avg}")
print(f"Sum of weights (returned=True): {sum_weights}")

Output:

Our array (a):
[[1 2 3]
 [4 5 6]]

Overall mean: 3.5
Mean along axis 0 (columns): [2.5 3.5 4.5]
Mean along axis 1 (rows): [2. 5.]

Weights: [1 2 3]
Weighted average (axis=1): [2.33333333 5.33333333]
Weighted avg (returned=True): [2.33333333 5.33333333]
Sum of weights (returned=True): [6. 6.]

Variance and Standard Deviation (numpy.var, numpy.std)

• numpy.var() or ndarray.var(): Computes the variance.
• numpy.std() or ndarray.std(): Computes the standard deviation.

Variance measures the average squared deviation from the mean. Standard deviation is the square root of the variance, providing a measure of spread in the same units as the data.

Important Parameter: ddof (Delta Degrees of Freedom). By default, ddof=0, calculating the population variance/standard deviation. For sample variance/standard deviation (dividing by N-1 instead of N), set ddof=1.

Example: Variance and Standard Deviation

import numpy as np

data = np.array([1, 2, 3, 4, 5])
print(f"Data: {data}\n")

# Population variance (ddof=0, default)
var_pop = np.var(data)
std_pop = np.std(data)
print(f"Population Variance (ddof=0): {var_pop}")
print(f"Population Std Dev (ddof=0): {std_pop}\n")

# Sample variance (ddof=1)
var_sample = np.var(data, ddof=1)
std_sample = np.std(data, ddof=1)
print(f"Sample Variance (ddof=1): {var_sample}")
print(f"Sample Std Dev (ddof=1): {std_sample}")

Output:

Data: [1 2 3 4 5]

Population Variance (ddof=0): 2.0
Population Std Dev (ddof=0): 1.4142135623730951

Sample Variance (ddof=1): 2.5
Sample Std Dev (ddof=1): 1.5811388300841898

Application: Statistical functions are fundamental for descriptive data analysis, understanding data distributions, feature engineering in machine learning, and quality control.

Find more functions and details in the NumPy statistics documentation.