IBM Data Science Test 2025 – 400 Free Practice Questions to Pass the Exam

When a learning rate is set too high, it can lead to oscillations or even divergence of the model during the training process. In gradient descent, the learning rate determines how much the weights of the model are updated in response to the estimated error each time the model's parameters are adjusted.

If the learning rate is excessively high, the updates to the model's weights can overshoot the optimal values. This can cause the model's loss function to fluctuate wildly instead of gradually converging to a minimum. Instead of settling down, the updates may repeatedly go back and forth across the optimal point, which manifests as oscillation. In extreme cases, the behavior can escalate into divergence, where the loss function increases indefinitely instead of reducing, indicating that the model is not learning effectively.

In contrast, a well-calibrated learning rate allows for gradual convergence toward the optimal parameters, improving the model's performance. Therefore, maintaining an appropriate learning rate is crucial for effective training of machine learning models.