Multiple Linear Regression
Multiple Linear Regression
Predicting a continuous value using more than one input feature.
What is Multiple Linear Regression?
Multiple Linear Regression (MLR) is a type of regression where the target variable (Y) is predicted using two or more independent variables (X₁, X₂, X₃, ...).
Simple vs Multiple Linear Regression
| Aspect | Simple Linear Regression | Multiple Linear Regression |
|---|---|---|
| Number of Inputs | One feature (X) | Two or more features (X₁, X₂, ..., Xₙ) |
| Equation | Y = b₀ + b₁X | Y = b₀ + b₁X₁ + b₂X₂ + ... + bₙXₙ |
| Graph | Straight line (2D) | Hyperplane (n-dimensional) |
| Use Case | Predict salary from experience | Predict house price from area, location, rooms |
Mathematical Formula
Where:
- Y = predicted output (dependent variable)
- b₀ = intercept (constant)
- b₁, b₂, ..., bₙ = coefficients (weights for each feature)
- X₁, X₂, ..., Xₙ = input features
- ε = error term
Real-Life Example
Predicting a House Price using:
- Area (sq. ft)
- Number of bedrooms
- Location score
- Age of the house
Key Assumptions of Multiple Linear Regression
Linearity
The relationship between input features and output is linear.
Independence
Observations are independent of each other.
Homoscedasticity
The variance of errors should be constant across all values of X.
Normality of Errors
Errors (residuals) should be normally distributed.
No Multicollinearity
Independent variables should not be highly correlated with each other.
How Does MLR Work?
Step-by-Step Process
- Collect and clean dataset.
- Choose multiple independent variables (X).
- Define the dependent variable (Y).
- Split data into train/test sets.
- Fit the regression model.
- Predict outputs on test data.
- Evaluate model using MSE, RMSE, R².
Python Example
pip install pandas numpy scikit-learnimport pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score
# Sample dataset
data = {
"Area": [1000, 1500, 2000, 2500, 3000, 3500, 4000],
"Bedrooms": [2, 3, 3, 4, 4, 5, 5],
"Age": [10, 5, 15, 8, 3, 2, 1],
"Price": [40, 55, 65, 80, 100, 120, 140] # in lakhs
}
df = pd.DataFrame(data)
# Features and target
X = df[["Area", "Bedrooms", "Age"]]
y = df["Price"]
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train model
model = LinearRegression()
model.fit(X_train, y_train)
# Predict
y_pred = model.predict(X_test)
# Evaluate
print("Coefficients:", model.coef_)
print("Intercept:", model.intercept_)
print("MSE:", mean_squared_error(y_test, y_pred))
print("R² Score:", r2_score(y_test, y_pred))Interpreting Coefficients
Each coefficient (b₁, b₂, ...) tells how much the output (Y) changes when that feature increases by 1 unit, keeping others constant.
| Feature | Coefficient | Meaning |
|---|---|---|
| Area | +0.03 | Price increases ₹0.03L per sq. ft |
| Bedrooms | +5 | Each extra room adds ₹5L |
| Age | -2 | Older houses cost ₹2L less per year |
Evaluation Metrics
| Metric | Description |
|---|---|
| MAE | Mean Absolute Error |
| MSE | Mean Squared Error |
| RMSE | Root Mean Squared Error |
| R² | Goodness of fit (0 to 1) |
| Adjusted R² | R² adjusted for number of predictors |
Problem: Multicollinearity
Multicollinearity occurs when two or more independent variables are highly correlated, making the model unstable.
How to detect:
- Correlation Matrix
- Variance Inflation Factor (VIF)
from statsmodels.stats.outliers_influence import variance_inflation_factor
import pandas as pd
vif = pd.DataFrame()
vif["Feature"] = X.columns
vif["VIF"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
print(vif)Real-Life Analogy
Multiple Linear Regression = Baking a Cake
The taste of a cake (Y) depends on multiple ingredients (X): flour, sugar, butter, eggs. Each ingredient contributes differently (coefficients). Adjusting them properly gives the perfect cake — just like the perfect prediction.
Real-World Applications
Real Estate
- Predict property prices
- Estimate rent based on location and size
E-Commerce
- Forecast sales
- Optimize pricing
Healthcare
- Predict patient recovery time
- Estimate medical costs
Finance
- Predict stock prices
- Credit risk modeling
Automotive
- Predict mileage
- Estimate maintenance cost
Weather
- Predict rainfall
- Estimate humidity from multiple sensors
Advantages
- Easy to interpret and implement.
- Captures relationships between multiple features and the target.
- Works well for many real-world structured datasets.
- Provides understanding through coefficients.
Disadvantages
Common Mistakes to Avoid
Best Practices
Quick Tips
- Always handle missing values before training.
- Encode categorical features properly.
- Check correlation matrix for multicollinearity.
- Use Adjusted R² for model selection.
- Visualize residuals to verify assumptions.
- Use cross-validation for stable performance.
Importance of MLR
Multi-Feature Modeling
- Captures real-world complexity
- Considers multiple inputs
Predictive Power
- Strong forecasting capability
- Used in business analytics
Foundation for Advanced ML
- Used in Ridge/Lasso regression
- Base of many ML models
Interpretability
- Coefficients reveal feature importance
- Easy for business users to understand
Golden Rule
Key Takeaway
Multiple Linear Regression extends Simple Linear Regression by using multiple input variables to predict a single continuous output. It's one of the most widely used ML techniques for real-world structured data problems — especially when you need both accuracy and interpretability.