Probabilistic pedotransfer functions for soil bulk density with calibrated uncertainty: a comparison of quantile regression and conformal prediction across 735 Brazilian catchments

Abstract

Soil bulk density is an essential soil physical characteristic needed to transform soil solute concentrations into areal or volumetric soil stocks of carbon, water and nutrients. Although bulk density is important, it is often omitted from soil surveys and is often estimated from soil survey data via pedotransfer functions (PTFs). While most current PTFs provide only one point estimate, however, none provide a statement of confidence, making them less useful in rigorously accounting for soil carbon and soil water, and less useful for downstream uncertainty propagation. The study designs and compares probabilistic PTFs to estimate soil bulk density based on soil texture, organic carbon content and bedrock depth in 735 catchments in Brazil from the CABra (openly available) database. The main contribution to this work is the quantification and calibration of prediction uncertainty, in addition to the point prediction accuracy. The models were: (1) ordinary linear regression, (2) random forest, (3) extremely randomized trees, (4) gradient boosting, (5) histogram gradient boosting, and (6) XGBoost. The gradient boosted model and the extremely random trees were clearly better than the linear baseline model with the best model (XGBoost) achieving a coefficient of determination (R²) of 0.84 and a root mean square error (RMSE) of 0.027 g cm⁻³ on the held-out test set. Organic carbon showed to be the most important variable, with a permutation importance of 0.53 and a Pearson correlation of −0.64 with bulk density, as is well known, and a positive correlation with the other variables. Two uncertainty quantification methods were then systematically compared: the first, gradient-boosting quantile regression, which yields prediction intervals that adapt to the input; and the second, split-conformal prediction, which provides distribution-free finite-sample bounds on prediction intervals. Split-conformal prediction was more well-calibrated on the held-out data as it had an 84.8% empirical coverage rate at the 90% nominal level, while quantile regression only achieved a 79.9% level of empirical coverage with similar or slightly wider intervals. No matter which method was used, the coverage was slightly under-coverage compared to the nominal coverage, which was explained to be due to the limited calibration sample size, and is discussed explicitly. This approach produces bulk-density estimates along with honest prediction intervals, statistically testable estimates of uncertainty, thus enabling rigorous propagation of uncertainty in soil carbon and hydrological accounting.