XGBoost, LightGBM, and CatBoost in Practice
Explore how gradient boosting models XGBoost LightGBM and CatBoost are used in practice to predict California house prices. Understand handling missing data training with early stopping limits and model regularization. Learn to choose appropriate libraries based on data size feature types and operational needs to build reliable regression models.
For California house-price prediction, a linear baseline can miss patterns such as location effects that change across income and population values. XGBoost fits many small regression trees in sequence, so later trees can correct residual patterns that earlier trees left behind.
The complete pipeline uses the California Housing dataset, divides its numeric rows into training, validation, and test sets, fits an XGBRegressor with the squared-error regression objective, and measures root mean squared error (RMSE) on validation data. The test set stays untouched while we choose training settings. Early stopping needs a separate validation set because it monitors validation performance while training. If we used the test set for that decision, the reported test score would no longer be an independent final check.
Fit a regression model with missing numeric values
The first model establishes that an XGBoost regressor can train on numeric tabular data containing missing entries. The code injects NaN values into one California Housing feature after the split, so the missing values belong to the feature data rather than the target house price.
The objective="reg:squarederror" setting tells the model to minimize squared prediction error for a regression target. RMSE reports prediction error in the target's units. In this dataset, the target is measured in hundreds of thousands of dollars, so an RMSE of