SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2305.14975 · EMNLP · 2023

Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback

Huaxiu Yao, Chelsea Finn, Christopher Manning, Allan Zhou, Eric Mitchell, Rafael Rafailov, Archit Sharma, Katherine Tian

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

A trustworthy real-world prediction system should produce well-calibrated confidence scores; that is, its confidence in an answer should be indicative of the likelihood that the answer is correct, enabling deferral to an expert in cases of low-confidence predictions. Recent studies have shown that unsupervised pretraining produces large language models (LMs) whose conditional probabilities are remarkably well-calibrated. However, the most widelyused LMs are fine-tuned with reinforcement learning from human feedback (RLHF-LMs), and some studies have suggested that RLHF-LMs produce conditional probabilities that are very poorly calibrated. In light of this perceived weakness, we conduct a broad evaluation of methods for extracting confidence scores from RLHF-LMs. For RLHF-LMs such as ChatGPT, GPT-4, and Claude, we find that verbalized confidences emitted as output tokens are typically better-calibrated than the model's conditional probabilities on the TriviaQA, SciQ, and TruthfulQA benchmarks, often reducing the expected calibration error by a relative 50%. * * Equal contribution. 1 i.e., the confidence in a prediction accurately reflects the probability that the prediction is correct (Guo et al., 2017).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2305.14975")
get_code_for_paper("2305.14975")
have("2305.14975")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2305.14975.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2305.14975.svg)](https://syntology.ai/paper/2305.14975)

Connect an agent — have() is free.