  Despite attractive theoretical guarantees and practical successes, Predictive
Interval (PI) given by Conformal Prediction (CP) may not reflect the
uncertainty of a given model. This limitation arises from CP methods using a
constant correction for all test points, disregarding their individual
uncertainties, to ensure coverage properties. To address this issue, we propose
using a Quantile Regression Forest (QRF) to learn the distribution of
nonconformity scores and utilizing the QRF's weights to assign more importance
to samples with residuals similar to the test point. This approach results in
PI lengths that are more aligned with the model's uncertainty. In addition, the
weights learnt by the QRF provide a partition of the features space, allowing
for more efficient computations and improved adaptiveness of the PI through
groupwise conformalization. Our approach enjoys an assumption-free finite
sample marginal and training-conditional coverage, and under suitable
assumptions, it also ensures conditional coverage. Our methods work for any
nonconformity score and are available as a Python package. We conduct
experiments on simulated and real-world data that demonstrate significant
improvements compared to existing methods.
