L1 Regularization Quiz: Test Your Knowledge

📚 Quick Study Guide

• 📏 L1 Regularization (Lasso): Adds a penalty term to the loss function equal to the absolute value of the magnitude of coefficients.
• 🧮 Formula: Loss Function + $\lambda \sum_{i=1}^{n} |\beta_i|$, where $\lambda$ is the regularization parameter and $\beta_i$ are the coefficients.
• 📉 Effect: Encourages sparsity, meaning some coefficients are driven to exactly zero.
• ✨ Feature Selection: Automatically performs feature selection by eliminating irrelevant features.
• 🤔 When to Use: Useful when you suspect that many features are irrelevant to the model.
• 🆚 L1 vs. L2: L1 regularization leads to sparse models, while L2 (Ridge) shrinks coefficients but rarely sets them exactly to zero.
• ⚙️ Tuning: The strength of the regularization is controlled by the hyperparameter $\lambda$ (lambda).

🧪 Practice Quiz

1. Which of the following is the primary effect of L1 regularization?

   1. A. Reducing model complexity by shrinking coefficients towards zero.
   2. B. Increasing model complexity by expanding the feature space.
   3. C. Preventing overfitting by adding more data.
   4. D. Improving model speed by using more CPU cores.

2. What is the mathematical formulation of the L1 regularization term added to the loss function?

   1. A. $\lambda \sum_{i=1}^{n} \beta_i^2$
   2. B. $\lambda \sum_{i=1}^{n} |\beta_i|$
   3. C. $\lambda \sqrt{\sum_{i=1}^{n} \beta_i^2}$
   4. D. $\lambda \sum_{i=1}^{n} (\beta_i - \mu)^2$

3. What type of feature selection does L1 regularization perform?

   1. A. Manual feature selection
   2. B. Recursive feature elimination
   3. C. Automatic feature selection
   4. D. Random feature selection

4. Under what circumstances is L1 regularization most appropriate?

   1. A. When all features are known to be relevant.
   2. B. When there are very few features in the dataset.
   3. C. When many features are suspected to be irrelevant.
   4. D. When high accuracy is not a priority.

5. What is the key difference between L1 and L2 regularization in terms of the resulting model?

   1. A. L1 leads to dense models, while L2 leads to sparse models.
   2. B. L1 leads to sparse models, while L2 leads to dense models.
   3. C. L1 only affects the intercept, while L2 affects coefficients.
   4. D. There is no significant difference between L1 and L2.

6. What does the hyperparameter $\lambda$ (lambda) control in L1 regularization?

   1. A. The learning rate of the model.
   2. B. The strength of the regularization penalty.
   3. C. The number of iterations in training.
   4. D. The type of optimization algorithm used.

7. Which of the following algorithms uses L1 regularization?

   1. A. Ridge Regression
   2. B. Lasso Regression
   3. C. Elastic Net Regression
   4. D. Both B and C