  Conformal prediction methodologies have significantly advanced the
quantification of uncertainties in predictive models. Yet, the construction of
confidence regions for model parameters presents a notable challenge, often
necessitating stringent assumptions regarding data distribution or merely
providing asymptotic guarantees. We introduce a novel approach termed CCR,
which employs a combination of conformal prediction intervals for the model
outputs to establish confidence regions for model parameters. We present
coverage guarantees under minimal assumptions on noise and that is valid in
finite sample regime. Our approach is applicable to both split conformal
predictions and black-box methodologies including full or cross-conformal
approaches. In the specific case of linear models, the derived confidence
region manifests as the feasible set of a Mixed-Integer Linear Program (MILP),
facilitating the deduction of confidence intervals for individual parameters
and enabling robust optimization. We empirically compare CCR to recent
advancements in challenging settings such as with heteroskedastic and
non-Gaussian noise.
