A data scientist is developing a regression model to predict housing prices. The initial model, a high-degree polynomial regression, achieves a near-perfect R-squared score on the training dataset but performs poorly on the validation dataset, exhibiting a significantly higher Mean Squared Error (MSE). This discrepancy indicates the model has high variance and is overfitting. Which of the following strategies, by directly modifying the loss function, is the most effective approach to minimize the model's variance and improve generalization?
Increase the polynomial degree of the model to better capture the complex underlying patterns in the data.
Apply k-means clustering to the feature set to generate a new categorical variable for the model.
Replace the Mean Squared Error (MSE) loss function with Mean Absolute Error (MAE) to reduce the impact of outliers.
Augment the Ordinary Least Squares (OLS) loss function with an L2 regularization term to penalize large coefficient values.
The correct answer involves augmenting the Ordinary Least Squares (OLS) loss function with an L2 regularization term. The scenario describes a classic case of overfitting, where the model is too complex and has high variance. L2 regularization (used in Ridge Regression) directly addresses this by adding a penalty term to the loss function that is proportional to the square of the magnitude of the model's coefficients. This penalizes large coefficients, effectively shrinking them and reducing the model's complexity. By introducing a small amount of bias, L2 regularization significantly reduces the model's variance, leading to better generalization on unseen data.
Replacing MSE with Mean Absolute Error (MAE) is a valid strategy for dealing with outliers but is not the primary method for reducing variance from model complexity. Applying k-means clustering is a feature engineering technique, not a direct modification of the loss function for variance control. Increasing the model's polynomial degree would further increase its complexity, worsening the existing high variance and overfitting problem.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is L2 regularization and how does it reduce overfitting?
Open an interactive chat with Bash
How does Mean Squared Error (MSE) compare to Mean Absolute Error (MAE) in addressing outliers?
Open an interactive chat with Bash
Why does increasing the polynomial degree worsen overfitting in regression models?