XGBoost Hyperparameters: Tuning the Learning Rate
Explore how tuning the learning rate hyperparameter in XGBoost affects model performance and convergence. Understand the trade-off between smaller learning rates and training rounds, and learn how to use validation AUC scores and iteration tracking to choose an optimal learning rate for better model accuracy.
We'll cover the following...
We'll cover the following...
Impact of learning rate on model performance
The learning rate is also referred to as eta in the XGBoost documentation, as well as step size shrinkage. This hyperparameter controls how much of a contribution each new estimator will make to the ensemble prediction. If you increase the learning rate, you may reach the optimal model, defined as having the highest performance on the validation set, ...