A data-science team is using RandomizedSearchCV to tune a GradientBoostingClassifier. The current search space samples the learning_rate hyperparameter uniformly between 0.01 and 0.30. After 100 sampled configurations, fewer than 10 candidates achieve acceptable AUC, and every high-performing candidate has a learning_rate below 0.06. The team wants to increase the probability of drawing competitive learning_rate values without increasing the number of search iterations or altering other hyperparameters. Which adjustment is most likely to improve the efficiency of their hyperparameter search?
Replace the random search with a grid search that tests 10 evenly spaced learning_rate values in the same range.
Sample learning_rate from a log-uniform distribution between 0.01 and 0.30 instead of a uniform distribution.
Keep learning_rate fixed at 0.1 and redirect the search budget to max_depth and n_estimators.
Increase the cross-validation folds in RandomizedSearchCV from 3 to 10 to reduce variance in AUC estimates.
Because learning_rate is a strictly positive, continuous hyperparameter that often spans several orders of magnitude, sampling it on a linear (uniform) scale spends most trials on the upper part of the interval and under-samples the smaller values where good models are clustered. Replacing the uniform sampling with a log-uniform distribution keeps the same bounds but allocates more probability mass to the lower decades (e.g., 0.01-0.05), making it far likelier that each random draw lands near the performant region without increasing n_iter. Fixing learning_rate at a single value removes it from the search entirely, exhaustive grid search would still devote most trials to sub-optimal high values, and increasing the number of CV folds affects variance estimation rather than the quality of hyperparameter samples.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
Why is a log-uniform distribution better than a uniform distribution for learning_rate in this case?
Open an interactive chat with Bash
What is the role of the learning_rate hyperparameter in GradientBoostingClassifier?
Open an interactive chat with Bash
How does RandomizedSearchCV improve hyperparameter tuning efficiency?