Scikit Learn - Conventions

Scikit-Learn - API Conventions

Scikit-Learn’s objects adhere to a uniform and consistent API, which generally consists of three complementary interfaces:

• Estimator interface: For building and fitting models from data.
• Predictor interface: For making predictions or inferring properties from fitted models.
• Transformer interface: For converting or preprocessing data.

These APIs adopt simple conventions, and their design choices aim to avoid unnecessary framework code, promoting ease of use and interoperability.

Purpose of Conventions

The conventions ensure that the API adheres to the following broad principles:

Consistency: All objects, whether basic (like LinearRegression) or composite (like Pipeline), share a consistent interface composed of a limited set of methods (e.g., fit(), predict(), transform()).

Inspection: Constructor parameters (hyperparameters) and parameter values learned by the algorithm (attributes ending with an underscore, e.g., coef_) are stored and exposed as public attributes, allowing easy introspection.

Non-proliferation of classes: Datasets are typically represented as NumPy arrays or SciPy sparse matrices. Hyperparameters are usually standard Python types like strings, numbers, or booleans. This minimizes the need for custom data structures.

Composition: Algorithms that can be expressed as sequences or combinations of transformations, or as meta-algorithms parameterized on other algorithms, are implemented and composed from existing building blocks (e.g., Pipeline, FeatureUnion, meta-estimators like GridSearchCV).

Sensible defaults: Whenever an operation requires a user-defined parameter, Scikit-Learn provides an appropriate default value. This default should enable the operation to be performed in a sensible way, often providing a reasonable baseline solution.

Key Conventions in Practice

Let’s explore some of these conventions with examples:

Type Casting and Data Format

Scikit-Learn estimators generally expect input data X to be numeric and will attempt to cast it to float64 (or float32 for some sparse matrix operations). The target y can be numeric or categorical depending on the task.

Example: Input data type casting.

import numpy as np
from sklearn.random_projection import GaussianRandomProjection

rng = np.random.RandomState(0)
X_float32 = rng.rand(10, 2000).astype(np.float32)
print(f"Original X dtype: {X_float32.dtype}")

transformer = GaussianRandomProjection(random_state=42)
X_new = transformer.fit_transform(X_float32)
print(f"Transformed X_new dtype: {X_new.dtype}")

Output:

Original X dtype: float32
Transformed X_new dtype: float64

In this example, X_float32 (dtype float32) is cast to float64 internally by fit_transform().

Refitting and Updating Parameters

Hyperparameters of an estimator can be updated after its instantiation using the set_params() method. This is particularly useful in model selection (e.g., with GridSearchCV). Calling fit() again after set_params() will refit the estimator with the new parameters.

Example: Changing the kernel of an SVC and refitting.

import numpy as np
from sklearn.datasets import load_iris
from sklearn.svm import SVC

X, y = load_iris(return_X_y=True)
clf = SVC(gamma='scale', random_state=0) # Initial gamma='scale' uses rbf kernel by default

# Change kernel to 'linear' and refit
clf.set_params(kernel='linear').fit(X, y)
predictions_linear = clf.predict(X[:5])
print(f"Predictions with linear kernel: {predictions_linear}")
print(f"Current kernel: {clf.kernel}")

# Change kernel back to 'rbf' (or another) and refit
clf.set_params(kernel='rbf', C=0.5).fit(X, y)
predictions_rbf = clf.predict(X[:5])
print(f"Predictions with RBF kernel (C=0.5): {predictions_rbf}")
print(f"Current kernel: {clf.kernel}, C: {clf.C}")

Output:

Predictions with linear kernel: [0 0 0 0 0]
Current kernel: linear
Predictions with RBF kernel (C=0.5): [0 0 0 0 0]
Current kernel: rbf, C: 0.5

This demonstrates how set_params() allows dynamic modification and refitting of estimators.

Multiclass and Multilabel Fitting

Scikit-Learn handles multiclass and multilabel classification tasks based on the format of the target data y provided to fit().

For multiclass classification, y is typically a 1D array of class labels.

Example: Multiclass classification with OneVsRestClassifier using 1D target array.

from sklearn.svm import SVC
from sklearn.multiclass import OneVsRestClassifier
from sklearn.preprocessing import LabelBinarizer

X_multi = np.array([[1, 2], [3, 4], [4, 5], [5, 2], [1, 1], [2,5]])
y_multi_1d = np.array([0, 0, 1, 1, 2, 1]) # 3 classes: 0, 1, 2

# Using OneVsRestClassifier with an SVC estimator
# gamma='scale' and random_state for reproducibility
classifier = OneVsRestClassifier(estimator=SVC(gamma='scale', random_state=0))
classifier.fit(X_multi, y_multi_1d)
predictions_1d_target = classifier.predict(X_multi)
print(f"Predictions (1D target): {predictions_1d_target}")

Output:

Predictions (1D target): [0 0 1 1 2 1]

Alternatively, for some classifiers or meta-estimators, y can be a 2D binary label indicator matrix (one column per class). LabelBinarizer can create this format.

y_multi_2d = LabelBinarizer().fit_transform(y_multi_1d)
print(f"Binarized y (first 3 rows):\n{y_multi_2d[:3]}")

classifier_2d_target = OneVsRestClassifier(estimator=SVC(gamma='scale', random_state=0))
classifier_2d_target.fit(X_multi, y_multi_2d)
predictions_2d_target = classifier_2d_target.predict(X_multi)
print(f"Predictions (2D target, first 3 rows):\n{predictions_2d_target[:3]}")

Output:

Binarized y (first 3 rows):
[[1 0 0]
 [1 0 0]
 [0 1 0]]
Predictions (2D target, first 3 rows):
[[1 0 0]
 [1 0 0]
 [0 1 0]]

For multilabel classification, where each instance can be assigned multiple labels, y must be a 2D binary indicator matrix. MultiLabelBinarizer is used for this.

Example: Multilabel classification.

from sklearn.preprocessing import MultiLabelBinarizer
# Sample multilabel data (list of lists/tuples of labels)
y_multilabel_list = [[0, 1], [0, 2], [1, 3], [0, 2, 3], [2, 4], [1,4]]
X_multilabel = X_multi # Reusing X_multi for simplicity

y_multilabel_2d = MultiLabelBinarizer().fit_transform(y_multilabel_list)
print(f"Binarized multilabel y (first 3 rows):\n{y_multilabel_2d[:3]}")

# Using the same OneVsRestClassifier setup for multilabel
classifier_multilabel = OneVsRestClassifier(estimator=SVC(gamma='scale', random_state=0))
classifier_multilabel.fit(X_multilabel, y_multilabel_2d)
predictions_multilabel = classifier_multilabel.predict(X_multilabel)
print(f"Predictions (multilabel, first 3 rows):\n{predictions_multilabel[:3]}")

Output:

Binarized multilabel y (first 3 rows):
[[1 1 0 0 0]
 [1 0 1 0 0]
 [0 1 0 1 0]]
Predictions (multilabel, first 3 rows):
[[1 1 0 0 0]
 [1 0 1 0 0]
 [0 1 0 1 0]]

These conventions make Scikit-Learn a versatile and user-friendly library. For a deeper dive into Scikit-Learn’s API and contributing guidelines, you can consult the developer documentation.