Multiple Choice Questions on Gradient Descent for High School Data Science

📚 Quick Study Guide

• 📈 Gradient Descent: An iterative optimization algorithm used to find the minimum of a function. Think of it like rolling a ball down a hill to the lowest point.
• 🧭 Learning Rate ($\alpha$): A parameter that determines the step size at each iteration while moving toward the minimum of a function. Too large, and you might overshoot; too small, and it takes forever!
• 📉 Cost Function (J): A function that measures the error between predicted values and actual values. The goal of gradient descent is to minimize this function. Common example: Mean Squared Error (MSE).
• 🔄 Iteration: Each step taken in the gradient descent process to update the parameters.
• 📐 Partial Derivatives: Gradient descent uses partial derivatives to determine the direction of steepest descent for each parameter.
• 🧮 Update Rule: The parameters are updated using the formula: $\theta_{new} = \theta_{old} - \alpha \cdot \nabla J(\theta)$, where $\theta$ represents the parameters, $\alpha$ is the learning rate, and $\nabla J(\theta)$ is the gradient of the cost function.
• 🛑 Convergence: The process stops when the cost function reaches a minimum or when the change in the cost function becomes very small.

Practice Quiz

1. Question 1: What is the primary goal of Gradient Descent?
   1. A. To maximize the cost function.
   2. B. To minimize the cost function.
   3. C. To find the average of the data.
   4. D. To normalize the data.
2. Question 2: What does the learning rate ($\alpha$) control in Gradient Descent?
   1. A. The number of iterations.
   2. B. The size of the steps taken towards the minimum.
   3. C. The initial value of the parameters.
   4. D. The type of cost function used.
3. Question 3: Which of the following is NOT a typical step in the Gradient Descent algorithm?
   1. A. Calculate the gradient of the cost function.
   2. B. Update the parameters based on the gradient.
   3. C. Initialize parameters randomly.
   4. D. Integrate the cost function.
4. Question 4: What happens if the learning rate is set too high?
   1. A. The algorithm converges faster.
   2. B. The algorithm may overshoot the minimum and fail to converge.
   3. C. The algorithm will always find the global minimum.
   4. D. The algorithm becomes more stable.
5. Question 5: What is the purpose of the cost function in Gradient Descent?
   1. A. To measure the performance of the algorithm.
   2. B. To determine the optimal learning rate.
   3. C. To measure the error between predicted and actual values.
   4. D. To normalize the input data.
6. Question 6: Which mathematical concept is heavily used in Gradient Descent to find the direction of steepest descent?
   1. A. Integration
   2. B. Differentiation
   3. C. Linear Regression
   4. D. Matrix Inversion
7. Question 7: What signifies that Gradient Descent has converged?
   1. A. The parameters reach infinity.
   2. B. The cost function stops decreasing significantly.
   3. C. The number of iterations reaches zero.
   4. D. The learning rate becomes very large.