[ML L12 N15] Regularization & Lasso Regression

If we have just one feature, the error is big, preformance is not good; but if we have too many features selected then it might be overfitting.

So we need to find a balance for how many features we want to select:

 

We can calculate the by Lasso Regression:

 

When we add more feature, we need to consider that it should have a bigger gain than the loss that I take as a result of having that additional feature in my regresssion.