Skip to content

Model calibration and confidence

Model calibration refers to the alignment between a model's predicted confidence and its actual accuracy. A well-calibrated model that assigns 80% confidence to a prediction is correct roughly 80% of the time; a miscalibrated model may be overconfident (assigning 80% confidence when it's only 60% accurate) or underconfident.

Calibration is distinct from accuracy: a model can be highly accurate but poorly calibrated, or vice versa. In high-stakes applications—medical diagnosis, fact-checking, legal decisions—poor calibration can be as harmful as low accuracy, because users rely on confidence estimates to know when to defer to human review.

Why calibration matters for misinformation work

Filtering at scale: Fact-checking organizations face high-velocity streams (thousands of claims per day on social media). A fact-checker must decide which claims to investigate. A poorly calibrated model will either waste effort on low-risk false positives or miss important claims due to high false negatives—leading to either resource drain or undetected misinformation.

Human-AI collaboration: When fact-checkers and LLMs work together, the human needs accurate confidence estimates to know when to trust the model vs. override it. Miscalibrated LLMs cause humans to either dismiss correct predictions or blindly follow incorrect ones.

Evidence-based decisions: Calibrated confidence intervals enable downstream systems (e.g., content moderation, misinformation warning systems) to make principled decisions about confidence thresholds for action.

Common calibration problems

Overconfidence: Modern deep neural networks and large language models are often overconfident. A model trained on in-distribution data may assign high confidence to out-of-distribution examples it has never seen. LLMs are particularly prone to hallucination paired with high confidence.

Underconfidence: Some models hedge excessively, assigning low confidence even to correct predictions—wasting the model's predictive signal.

Class imbalance: When data is imbalanced (e.g., 95% of claims are non-check-worthy), models may be well-calibrated on the majority class but miscalibrated on the minority class.

Domain shift: A model calibrated on training data may become miscalibrated when deployed to a different domain or distribution.

Calibration techniques

Post-hoc scaling (parametric): - Platt scaling: Fit a logistic regression on model outputs to map predicted confidence to true accuracy. - Temperature scaling: Adjust the "temperature" parameter of a softmax to change confidence without retraining. - Isotonic regression: Non-parametric scaling that fits arbitrary monotonic mappings.

Limitation: These methods fit a global mapping and struggle with input-dependent or class-dependent miscalibration.

Residual correction (non-parametric): - NN-PPI (Nearest-Neighbor Prediction-Powered Inference) — applies a local residual correction learned from semantically similar labeled examples, addressing input-dependent bias without global assumptions.

Advantage: More flexible than parametric scaling; works well when miscalibration is localized. Limitation: Requires a small labeled calibration set.

Confidence intervals: - PPI and variants — provide \((1-\alpha)\) confidence intervals around model predictions using labeled residuals, enabling uncertainty quantification for downstream decisions.

Retraining / fine-tuning: - Simply retraining the model on in-distribution data, weighted to match the deployment distribution, can improve calibration. Limitation: expensive and often infeasible when retraining data is unavailable.

Evaluation

A calibrated model's expected calibration error (ECE) is small: averaging the gap between predicted confidence and true accuracy across confidence bins yields a low value. Common metrics:

  • Expected Calibration Error (ECE): binwise average of |confidence − accuracy|
  • Maximum Calibration Error (MCE): worst-case gap
  • Brier score: mean squared difference between predicted probabilities and true labels
  • Log loss: negative log-likelihood of true labels under predicted probabilities
  • Reliability diagrams: plots of predicted confidence vs. observed accuracy; well-calibrated models lie on the diagonal

Connections

Key papers