Feature Selection

Choose the most important features for your model.

• Use correlation, variance, or importance scores.
• Remove irrelevant or redundant features.

import pandas as pd
from sklearn.feature_selection import SelectKBest, f_classif

X = df.drop("Target", axis=1)
y = df["Target"]

selector = SelectKBest(score_func=f_classif, k=5)
X_new = selector.fit_transform(X, y)
print("Selected features:", X.columns[selector.get_support()])

Feature Extraction

Create new meaningful features from existing data.

• Combine columns (e.g., Speed = Distance / Time).
• Aggregate values (mean per group).

df["Speed"] = df["Distance"] / df["Time"]
df["AverageMarks"] = df.groupby("Class")["Marks"].transform("mean")

Feature Transformation

Apply mathematical operations to improve data quality.

• Log, Square, Square Root transformations.
• Used to reduce skewness in data.

import numpy as np

df["LogSalary"] = np.log(df["Salary"] + 1)
df["SqrtAge"] = np.sqrt(df["Age"])

$$ X_{log} = \log(X + 1) $$

Encoding Categorical Variables

Convert text categories to numeric form.

• Label Encoding: Assigns numeric labels.
• One-Hot Encoding: Creates binary columns.
• Ordinal Encoding: Maintains order in categories.

from sklearn.preprocessing import LabelEncoder
import pandas as pd

encoder = LabelEncoder()
df["Gender_Code"] = encoder.fit_transform(df["Gender"])

# One-Hot Encoding
df = pd.get_dummies(df, columns=["City"], drop_first=True)

Feature Scaling

Bring all features to the same scale.

• Standardization: Mean = 0, SD = 1.
• Normalization: Scales to [0, 1].

from sklearn.preprocessing import StandardScaler, MinMaxScaler

scaler = StandardScaler()
df[["Age", "Salary"]] = scaler.fit_transform(df[["Age", "Salary"]])

scaler2 = MinMaxScaler()
df[["Score"]] = scaler2.fit_transform(df[["Score"]])

$$ z = \frac{x - \mu}{\sigma}, \quad x_{norm} = \frac{x - x_{min}}{x_{max} - x_{min}} $$

Handling Date & Time Features

Extract useful insights from timestamps.

import pandas as pd

df["Date"] = pd.to_datetime(df["Date"])
df["Year"] = df["Date"].dt.year
df["Month"] = df["Date"].dt.month
df["Weekday"] = df["Date"].dt.day_name()
df["IsWeekend"] = df["Weekday"].isin(["Saturday", "Sunday"])

Binning (Discretization)

Convert continuous data into categories.

import pandas as pd

df["AgeGroup"] = pd.cut(df["Age"], 
                        bins=[0, 18, 35, 60, 100], 
                        labels=["Child", "Young", "Adult", "Senior"])

Polynomial Features

Capture non-linear relationships.

from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(df[["Hours"]])

$$ y = b_0 + b_1x + b_2x^2 + b_3x^3 + ... $$

Interaction Features

Combine multiple features into one.

df["Income_per_Age"] = df["Income"] / df["Age"]
df["BMI"] = df["Weight"] / (df["Height"]**2)

Dimensionality Reduction

Reduce number of features while keeping information.

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)