0130 秒速览:这份文档在说什么

01The 30-second version

先讲一个场景。一个客服 Agent 回答了用户关于处方剂量的问题,模型报告自己「0.92 置信」。答案是错的。事后复盘发现:在这家公司自己的标注留出集上,模型说 0.9 的那批回答,实际准确率只有 64%。上线前没有人测过校准。而升级规则写的是「置信度低于 0.5 就转人工」——可模型几乎从来不报低于 0.7 的数。

Start with a scene. A support agent answers a question about prescription dosage. The model reports 0.92 confidence. The answer is wrong. The post-mortem finds that on the company's own labelled hold-out set, answers where the model said 0.9 were correct only 64% of the time. Nobody measured calibration before launch. And the escalation rule read "escalate if confidence is below 0.5" — while the model almost never reported below 0.7.

整份文档要建立的,是下面这条流水线。你会发现它每一次都长成同一个形状:采一个原始信号 → 在留出标注集上量出它偏了多少 → 拟合一个校准器把它映射成真正的概率 → 用这个概率设门槛 → 分布漂移时重新校准。

What this document builds is the pipeline below. It has the same shape every time: collect a raw signal → measure how miscalibrated it is on a held-out labelled set → fit a calibrator that maps it to a real probability → set your threshold on that probability → recalibrate when the distribution shifts.

02为什么「模型自己说的置信度」是表演

02Why the model's stated confidence is theatre

Future AGI 的工程博客把三个结构性原因梳理得很清楚(以下为该文观点,非官方规范):

Future AGI's engineering blog lays out three structural reasons (these are that article's arguments, not an official specification):

原因一:预训练根本没教过校准

Reason 1: pretraining never taught calibration

模型被训练去预测下一个 token,不是去预测「我这个答案对不对」。它内部的 token 概率对「下一个 token 是什么」这件事是校准良好的——但「我有 90% 把握」这句话本身,只是模型基于训练语料里置信话术的分布,选出来的一串词。它和底层论断的内部概率之间,没有直接连接。

The model is trained to predict the next token, not to predict whether the resulting answer is right. Its internal token probabilities are well calibrated for the next-token task — but the sentence "I'm 90% sure" is itself a token sequence chosen from whatever confidence-language patterns appeared in training. It has no direct link to the internal probability of the underlying claim.

原因二:RLHF 放大了过度自信

Reason 2: RLHF amplifies overconfidence

奖励模型偏好听起来果断的回答。模型学到「含糊其辞会被扣分」,于是就比它应该的更少地含糊。Tian 等人在 EMNLP 2023 的工作(《Just Ask for Calibration》)及后续研究显示,跨 GPT-4、Claude、Llama-2 系列,自述置信度与准确率单调相关但系统性偏高,而且偏差最大的区间正是 0.7–1.0 ——恰恰是高风险、高置信决策发生的地方。

Reward models prefer answers that sound decisive. The model learns that hedging is penalised, so it hedges less than it should. Tian et al. (EMNLP 2023, "Just Ask for Calibration") and follow-up work show that across the GPT-4, Claude and Llama-2 families, verbalized confidence is monotonically related to accuracy but systematically biased upward, and the gap is largest in the 0.7–1.0 region — exactly where high-stakes, high-confidence decisions live.

原因三:自述是「事后」的

Reason 3: the self-report is post-hoc

当你问「你有多确定」时,答案已经写在纸上了。模型这时生成的数字,只是与 prompt 给的评分标准保持一致,不是对正确性的内部估计。换一套 prompt 措辞,你会得到一套不同的刻度。

By the time you ask "how sure are you," the answer is already on the page. The number the model then generates is consistent with whatever rubric your prompt supplied — not with an internal estimate of correctness. A different prompt produces a different scale.

03三种能用的信号

03Three signals you can actually use

读者读到这里的自然疑问是:那我用什么?公开可得的替代信号主要有两类——模型内部的 token 概率,和多次采样之间的分歧。再加上第三条路:直接训一个小分类器。

The natural next question: so what do I use instead? Two families of signal are publicly available — the model's internal token probabilities, and disagreement across repeated samples. Plus a third route: train a small classifier directly.

信号 A:logprob 聚合(API 暴露 token 概率时)

Signal A: logprob aggregation (when the API exposes token probabilities)

OpenAI 的 chat completions 端点在传 logprobs=True 时会返回 top-k 对数概率;多数开放权重推理服务(vLLM、TGI、llama.cpp)默认就暴露;Anthropic 在部分端点上返回。能看到 logprob 时,两个派生信号最有用:

OpenAI's chat completions endpoint returns top-k log-probabilities when you pass logprobs=True; most open-weight inference servers (vLLM, TGI, llama.cpp) expose them by default; Anthropic returns them on certain endpoints. When logprobs are visible, two derived signals dominate:

• 承载论断的 token 上的平均 top-1 logprob——只对实体、数字、人名这些真正携带语义的位置求平均。
• top-k token 熵——每一步 top-k 分布的香农熵。熵高意味着模型在那个位置在几个都说得通的 token 之间犹豫;论断 token 上的熵尖峰,与错误强相关。

• Mean top-1 logprob over claim-bearing tokens — averaged only over entity, number and name positions that actually carry meaning.
• Top-k token entropy — Shannon entropy over the top-k distribution at each step. High entropy means the model was choosing between several plausible tokens at that position; entropy spikes on claim tokens correlate strongly with error.

import math

def claim_token_uncertainty(tokens, claim_mask):
    # tokens: [{token, top_logprob, top_k=[(tok, logp), ...]}, ...]
    # claim_mask: [bool, ...] —— True 表示这个位置承载论断
    logps, entropies = [], []
    for tok, is_claim in zip(tokens, claim_mask):
        if not is_claim:
            continue
        logps.append(tok["top_logprob"])
        probs = [math.exp(lp) for _, lp in tok["top_k"]]
        z = sum(probs)
        probs = [p / z for p in probs]
        entropies.append(-sum(p * math.log(p + 1e-12) for p in probs))
    return {
        "mean_logprob": sum(logps) / max(len(logps), 1),
        "mean_entropy": sum(entropies) / max(len(entropies), 1),
    }
# 代码来源:Future AGI 工程博客(2026-02-24 发布,2026-05-20 更新)

import math

def claim_token_uncertainty(tokens, claim_mask):
    # tokens: [{token, top_logprob, top_k=[(tok, logp), ...]}, ...]
    # claim_mask: [bool, ...] — True at claim-bearing positions
    logps, entropies = [], []
    for tok, is_claim in zip(tokens, claim_mask):
        if not is_claim:
            continue
        logps.append(tok["top_logprob"])
        probs = [math.exp(lp) for _, lp in tok["top_k"]]
        z = sum(probs)
        probs = [p / z for p in probs]
        entropies.append(-sum(p * math.log(p + 1e-12) for p in probs))
    return {
        "mean_logprob": sum(logps) / max(len(logps), 1),
        "mean_entropy": sum(entropies) / max(len(entropies), 1),
    }
# Source: Future AGI engineering blog (published 2026-02-24, updated 2026-05-20)

信号 B:语义熵(API 隐藏 logprob 时)

Signal B: semantic entropy (when logprobs are hidden)

2026 年多数商用托管端点不返回 logprob。这时的办法是从采样行为里反推不确定性。

Most hosted commercial endpoints in 2026 do not return logprobs. The workaround is to infer uncertainty from sampling behaviour.

朴素做法是词面熵:在 temperature 0.7 下采 N 条回答,看字符串重合度。问题在于同义改写会被算成分歧——「巴黎是首都」和「首都是巴黎」在字面上完全不同,意思却一样。

The naive version is lexical entropy: sample N completions at temperature 0.7 and measure surface-string agreement. The problem is that paraphrases register as disagreement — "Paris is the capital" and "The capital is Paris" look distinct while expressing the same answer.

语义熵(Farquhar 等,《Nature》2024)解决了这个问题:采 N 条,按意思聚类,然后对簇的占比而不是原始字符串算熵。两种聚类方式:

Semantic entropy (Farquhar et al., Nature 2024) fixes this: sample N completions, cluster them by meaning, and compute entropy over the cluster proportions rather than the raw strings. Two ways to cluster:

• 双向 NLI:两条回答互相蕴含,就归为一簇。算力更重,在改写密集的任务上更准。
• 先嵌入再聚类:用句向量模型编码,再用 HDBSCAN 或层次聚类在调好的阈值上分簇。更便宜,边界情况更松。

• Bidirectional NLI: two completions join the same cluster if a small NLI model says each entails the other. Heavier compute, more accurate on paraphrase-dense tasks.
• Embed then cluster: encode with a sentence encoder, then cluster with HDBSCAN or agglomerative clustering at a tuned threshold. Cheaper, looser at the edges.

import math

def semantic_entropy(samples, equivalent_fn):
    # samples: 生成的字符串列表
    # equivalent_fn(a, b) -> bool: 双向 NLI 或嵌入距离判等
    clusters = []
    for s in samples:
        placed = False
        for cluster in clusters:
            if equivalent_fn(s, cluster[0]):
                cluster.append(s); placed = True; break
        if not placed:
            clusters.append([s])
    n = len(samples)
    probs = [len(c) / n for c in clusters]
    return -sum(p * math.log(p + 1e-12) for p in probs)
# 代码来源:Future AGI 工程博客

import math

def semantic_entropy(samples, equivalent_fn):
    # samples: list of generated strings
    # equivalent_fn(a, b) -> bool: bidirectional NLI or embedding-distance check
    clusters = []
    for s in samples:
        placed = False
        for cluster in clusters:
            if equivalent_fn(s, cluster[0]):
                cluster.append(s); placed = True; break
        if not placed:
            clusters.append([s])
    n = len(samples)
    probs = [len(c) / n for c in clusters]
    return -sum(p * math.log(p + 1e-12) for p in probs)
# Source: Future AGI engineering blog

信号 C:训练一个分类头

Signal C: a trained classifier head

如果你有足够的数据,最准的做法是直接在 (输入, 是否正确) 配对上训一个小分类器。它是负载专属的——只在你自己的流量分布上有效,但正因为如此,在数据够的时候准确率最高。

With enough data, the most accurate route is to train a small classifier directly on (input, was_correct) pairs. It is workload-specific — valid only on your own traffic distribution — but for exactly that reason it gives the highest accuracy once you have the data.

04怎么衡量「偏了多少」:Brier、ECE、可靠性图

04Measuring how far off you are: Brier, ECE, reliability diagrams

有了原始信号只是第一步。一个信号只有在你知道它偏了多少的时候才有用。这一节讲三个工具,它们其实是同一件事的三种呈现。

Having a raw signal is only step one. A signal is useful only once you know how miscalibrated it is. This section covers three tools, which are three views of the same thing.

金标集:整条流水线的地基

The gold set: the foundation of the whole pipeline

上面三个指标都要在同一样东西上算——一个留出标注集。建议做法:

All three metrics are computed on the same thing: a held-out labelled set. Recommended practice:

1. 规模 500–2000 条,且必须是生产代表性的样本。
2. 从生产 trace 里捞,不要自己编测试用例。编出来的用例分布和真实流量不是一回事。
3. 向最难的 10% 倾斜——边界情况、罕见意图。门控恰恰是在这些地方失效的。
4. 人工打标。这是金标集,下游一切都依赖它。

1. 500 to 2000 examples, and they must be production-representative.
2. Pull from production traces, don't invent test cases. Invented cases don't share the distribution of real traffic.
3. Skew toward the hardest 10% — edge cases, rare intents. That's precisely where gates fail.
4. Label by hand. This is the gold set; everything downstream depends on it.

05后处理校准:Platt 缩放与保序回归

05Post-hoc calibration: Platt scaling and isotonic regression

量出偏差之后,下一个自然的问题是:怎么修?好消息是不用重训模型——两个标准的后处理方法就能把一个歪掉的信号映射成可用的概率。

Once you've measured the bias, the next question is how to fix it. The good news: you don't retrain the model. Two standard post-hoc methods map a skewed signal onto a usable probability.

Platt 缩放

Platt scaling

在校准划分上,对 (原始信号, 是否正确) 配对拟合一个单参数逻辑回归。

Fit a one-parameter logistic regression on (raw_signal, was_correct) pairs from a calibration split.

import numpy as np
from sklearn.linear_model import LogisticRegression

def fit_platt(raw_signals, labels):
    X = np.array(raw_signals).reshape(-1, 1)
    y = np.array(labels)
    return LogisticRegression().fit(X, y)

def apply_platt(model, raw_signal):
    return model.predict_proba([[raw_signal]])[0, 1]

适用于偏差是单调的、在 logit 空间大致线性的情况。拟合便宜,200 条以上的小校准集就稳定,是默认起点。

Works when miscalibration is monotonic and roughly linear in logit space. Cheap to fit, stable on 200+ examples, and the default starting point.

保序回归(isotonic regression)

Isotonic regression

拟合一个非参数的单调映射,能处理更奇怪的、非线性的校准曲线。

Fits a non-parametric monotonic mapping, handling weirder, non-linear calibration curves.

from sklearn.isotonic import IsotonicRegression

def fit_isotonic(raw_signals, labels):
    return IsotonicRegression(out_of_bounds="clip").fit(raw_signals, labels)

需要更多数据(稳定通常要 1000 条以上),但不做参数假设、直接从数据读出形状。当 Platt 在尾部修不够时,就换它。

Needs more data (1000+ examples for stability) but reads the shape off the data with no parametric assumptions. Switch to it when Platt under-corrects in the tails.

06Agent 特有的难题:不确定性会沿轨迹累积

06The agent-specific problem: uncertainty compounds along a trajectory

到这里为止讲的都是单轮问答的校准。但 Agent 不是单轮的——它调工具、读结果、再决策,一步接一步。这带来一个全新的问题:第 5 步的不确定性,有多少是它自己的,有多少是从前 4 步继承来的?

Everything so far applies to single-turn question answering. But agents are not single-turn — they call a tool, read the result, decide again, step after step. That creates a new problem: how much of the uncertainty at step 5 is its own, and how much was inherited from steps 1 through 4?

两个开创性框架

Two pioneering frameworks

UProp(arXiv 2506.17419)把序贯决策的不确定性拆成两部分:内在不确定性(当前这一步决策本身固有的)和外来不确定性——一个互信息量,描述应当从前面各步继承多少不确定性。它通过对轨迹相关采样的逐点互信息取期望,来近似决策分布之间的互信息。

UProp (arXiv 2506.17419) decomposes sequential decision uncertainty into two parts: internal uncertainty intrinsic to the current decision, and extrinsic uncertainty — a mutual-information quantity describing how much uncertainty should be inherited from preceding decisions. It approximates the mutual information between decision distributions by taking an expectation of pointwise mutual information over trajectory-dependent samplings.

SAUP(Situation Awareness Uncertainty Propagation,arXiv 2412.01033)则引入情境感知:用插件式估计器给每一步附上不确定性估计,再通过基于 HMM 的情境权重做传播——也就是说,不同处境下的同一步,权重不一样。

SAUP (Situation Awareness Uncertainty Propagation, arXiv 2412.01033) adds situational awareness: it attaches per-step uncertainty estimates using plug-in estimators and propagates them via HMM-based situational weights — meaning the same step carries different weight in different situations.

多 Agent:分歧不等于不确定

Multi-agent: disagreement is not the same as uncertainty

当你用一组 Agent 投票时,直觉是「大家意见不合 = 不确定」。但研究指出这个直觉会双向出错:

When you have a panel of agents vote, the intuition is "they disagree, so it's uncertain." Research points out this intuition fails in both directions:

• 良性多样性会造成信心不足——大家从不同角度给出等价的正确答案,却被算成分歧。
• 沟通诱发的一致会造成过度自信——Agent 之间互相看见对方的答案后趋同,这种一致不是独立证据。

• Benign diversity causes under-confidence — several equivalent correct answers from different angles get counted as disagreement.
• Communication-induced agreement causes over-confidence — agents converge after seeing each other's answers, and that agreement is not independent evidence.

目前有三类校准路线:后处理多数派校准器(用投票或熵信号)、LLM 自陈置信度估计器(训练模型报告校准过的置信度)、训练式校准器(从置信度与分歧特征估计正确性)。DiscoUQ-LLM 从分歧特征学习,MATU 用张量分解建模不确定性——都属于这一支。

Three calibration routes currently exist: post-hoc plurality calibrators (using voting or entropy signals), LLM-elicited confidence estimators (training the model to report calibrated confidence), and trained calibrators (estimating correctness from confidence and disagreement features). DiscoUQ-LLM learns from disagreement features and MATU models uncertainty via tensor decomposition — both belong to this last group.

共形预测:给弃答一个可证明的保证

Conformal prediction: a provable guarantee on abstention

2026 年的几项工作把它推向 Agent 场景:

Several 2026 works push it into agent settings:

• 角色分层的共形风险控制(arXiv 2607.24343):不用聚合风险,而是按工具调用的角色分层做风险控制——因为「查天气」和「发起转账」的可容忍错误率显然不一样。
• PASC(arXiv 2605.18812):面向多阶段流水线的联合覆盖保证,把多阶段问题归约成对「联合最大不符合度分数」的单标量共形问题。作者指出这个归约直接适用于复合 LLM 系统与 Agent 流水线。
• 可证明对齐保证的不确定性感知弃答(arXiv 2607.04430):分裂共形预测提供分布无关的覆盖保证;与其构造预测集,不如做弃答感知的点预测——系统要么返回一个回答,要么弃答。
• 另有工作把共形风险控制应用到移动端 GUI Agent(CORA)。

• Role-stratified conformal risk control (arXiv 2607.24343): rather than aggregate risk, stratify risk control by the role of the tool call — the tolerable error rate for "look up the weather" and "initiate a transfer" is obviously not the same.
• PASC (arXiv 2605.18812): joint coverage guarantees for multi-stage pipelines, reducing the multi-stage problem to a single scalar conformal problem on the joint maximum nonconformity score. The authors note this reduction applies directly to compound LLM systems and agent pipelines.
• Uncertainty-aware abstention with provable alignment guarantees (arXiv 2607.04430): split conformal prediction gives distribution-free coverage; instead of building a prediction set, do abstention-aware point prediction — the system either returns one answer or abstains.
• Related work applies conformal risk control to mobile GUI agents (CORA).

07落地:三件套与升级门控

07In production: the three pieces and the gate

把整套东西接进生产,只需要三块。这三块也正是最容易漏掉其中一块的地方——漏掉哪一块,整条链子都断。

Wiring all of this into production takes three pieces. It's also where teams most often ship two out of three — and any one missing breaks the chain.

① span 埋点:把信号记进 trace

1. Span instrumentation: put the signals on the trace

不确定性信号应该作为自定义 span 属性,和标准的 OpenTelemetry GenAI 字段并排记录。这样它们天然跟着 trace 树走,后面做分析、聚类、回归都有数据。典型的三个属性:llm.semantic_entropy、llm.mean_top_logprob、llm.mean_entropy。

Uncertainty signals belong on the trace as custom span attributes, alongside the standard OpenTelemetry GenAI fields. That way they ride the trace tree and you have data for later analysis, clustering and regression testing. Three typical attributes: llm.semantic_entropy, llm.mean_top_logprob, llm.mean_entropy.

with tracer.start_as_current_span("llm.answer_with_uncertainty") as span:
    samples = [client.chat.completions.create(
        model="...", messages=[...], temperature=0.7,
        logprobs=True, top_logprobs=5,
    ) for _ in range(n_samples)]

    sem_entropy = semantic_entropy(
        [s.choices[0].message.content for s in samples],
        equivalent_fn=nli_equivalent)
    logp_signal = claim_token_uncertainty(
        extract_tokens(samples[0]), claim_mask=mask_claims(samples[0]))

    span.set_attribute("llm.semantic_entropy", sem_entropy)
    span.set_attribute("llm.mean_top_logprob", logp_signal["mean_logprob"])
    span.set_attribute("llm.mean_entropy",    logp_signal["mean_entropy"])
    span.set_attribute("llm.n_samples",       n_samples)
    return majority_answer(samples), sem_entropy, logp_signal
# 结构参考:Future AGI traceAI 示例(Apache 2.0,OpenTelemetry 原生)

with tracer.start_as_current_span("llm.answer_with_uncertainty") as span:
    samples = [client.chat.completions.create(
        model="...", messages=[...], temperature=0.7,
        logprobs=True, top_logprobs=5,
    ) for _ in range(n_samples)]

    sem_entropy = semantic_entropy(
        [s.choices[0].message.content for s in samples],
        equivalent_fn=nli_equivalent)
    logp_signal = claim_token_uncertainty(
        extract_tokens(samples[0]), claim_mask=mask_claims(samples[0]))

    span.set_attribute("llm.semantic_entropy", sem_entropy)
    span.set_attribute("llm.mean_top_logprob", logp_signal["mean_logprob"])
    span.set_attribute("llm.mean_entropy",    logp_signal["mean_entropy"])
    span.set_attribute("llm.n_samples",       n_samples)
    return majority_answer(samples), sem_entropy, logp_signal
# Structure adapted from the Future AGI traceAI example (Apache 2.0, OpenTelemetry-native)

② 标注集:锚定校准

2. The labelled set: the calibration anchor

见上一节的金标集做法。这里补一条评测视角:除了「答案对不对」,你还要评「该升级的时候有没有升级」。可以用现成的评测模板(如 AnswerRefusal、TaskCompletion、Groundedness)组合,再加一条校准专用的评分标准,例如:

See the gold-set practice in the previous section. One addition from the evaluation side: besides "was the answer correct," you also need to score "did it escalate when it should have." Combine off-the-shelf templates (AnswerRefusal, TaskCompletion, Groundedness) with a calibration-specific rubric, for example:

③ 门控:把校准后的概率变成阈值

3. The gate: turn the calibrated probability into a threshold

阈值是业务决策,校准是工程。医疗分诊和内部聊天机器人显然应该落在曲线上不同的操作点。工程侧的责任只是保证:那个 0.65 确实意味着「65% 会对」。

The threshold is a business decision; the calibration is engineering. Medical triage and an internal chatbot obviously operate at different points on the curve. Engineering's job is only to guarantee that 0.65 really does mean "correct 65% of the time."

def route(prompt):
    answer, sem_ent, logp = answer_with_uncertainty(prompt)
    raw = combine(sem_ent, logp["mean_logprob"], verbalized_confidence(answer))
    p_correct = platt.predict(raw)          # 校准后的概率
    if p_correct < 0.65:                    # 阈值 = 业务决策
        return escalate_to_human(prompt, reason=f"p_correct={p_correct:.2f}")
    return answer
# 结构参考:Future AGI 工程博客

def route(prompt):
    answer, sem_ent, logp = answer_with_uncertainty(prompt)
    raw = combine(sem_ent, logp["mean_logprob"], verbalized_confidence(answer))
    p_correct = platt.predict(raw)          # calibrated probability
    if p_correct < 0.65:                    # threshold = business decision
        return escalate_to_human(prompt, reason=f"p_correct={p_correct:.2f}")
    return answer
# Structure adapted from the Future AGI engineering blog

08对比与选型:什么情况用哪个

08Choosing an approach: what to use when

09常见坑与限制

09Pitfalls and limits

诚实的边界

Honest limits

• 多步 Agent 的 UQ 仍然研究不足。综述明确这么说。UProp / SAUP 是开创性工作,不是成熟工具链——不要指望装个库就解决。
• 本文的生产实践部分,主要来源是一篇厂商工程博客(Future AGI,2026-02-24 发布 / 2026-05-20 更新)。其中的方法论(logprob 聚合、语义熵、Brier/ECE、Platt)有独立学术来源支撑,但「ECE 降低 30%–60%」「ECE < 0.05 是合理目标」这类具体数字,截至调研时仅见于该单一来源,未能交叉验证,请当作经验参考而非基准。
• Farquhar 等的语义熵结论是在问答基准上得出的,迁移到工具调用 Agent 的效果未见系统性公开评测。
• 共形方法给的是长期覆盖率保证,不是对单条回答的保证——单次它仍然可能错。

• UQ for multi-step agents remains underexplored. The survey says so explicitly. UProp and SAUP are pioneering work, not a mature toolchain — don't expect to install a library and be done.
• The production-practice sections here rest largely on one vendor engineering blog (Future AGI, published 2026-02-24, updated 2026-05-20). Its methodology (logprob aggregation, semantic entropy, Brier/ECE, Platt) has independent academic backing, but specific figures like "cuts ECE by 30–60%" and "ECE below 0.05 is a reasonable target" appear, as of this research date, only in that single source and could not be cross-verified. Treat them as experience-based guidance, not benchmarks.
• Farquhar et al.'s semantic-entropy result was established on QA benchmarks; no systematic public evaluation of its transfer to tool-calling agents was found.
• Conformal methods give long-run coverage guarantees, not guarantees on any single answer — an individual response can still be wrong.

10术语表

10Glossary

11来源清单与延伸阅读

11Sources and further reading

下列来源为 2026-08-27 调研时实际检索/阅读到的材料。标签含义:论文同行评审或预印本研究;文档官方文档或规范;厂商/社区厂商工程博客或社区观点,属经验之谈而非权威事实。

The sources below were retrieved and read during research on 2026-08-27. Tags: paper peer-reviewed or preprint research; docs official documentation or specification; vendor vendor engineering blog or community opinion — experience, not authoritative fact.

• 厂商Nikhil Pareek,《Evaluating LLM Confidence and Uncertainty (2026): The Calibration Methodology》,Future AGI 博客,2026-02-24 发布 / 2026-05-20 更新 —— 链接(本文生产实践、代码示例与 ECE 数字的主要来源)
• 论文《Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities》,arXiv 2602.05073 —— 链接
• 论文Farquhar, S. 等,《Detecting hallucinations in large language models using semantic entropy》,Nature(2024)
• 论文Tian, K. 等,《Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback》,EMNLP(2023)
• 论文Phillips, E.、Gustafsson, F. K.、Wu, S.、Thakur, A.、Clifton, D. A.,《Entropy Alone is Insufficient for Safe Selective Prediction in LLMs》,arXiv 2603.21172,2026-03-22 —— 链接
• 论文《UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making》,arXiv 2506.17419 —— 链接
• 论文《SAUP: Situation Awareness Uncertainty Propagation on LLM Agent》,arXiv 2412.01033 —— 链接
• 论文《Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls》,arXiv 2607.24343 —— 链接(调研时 fetch 遇到限流,内容依据搜索摘要,标注为未完整核实)
• 论文《PASC: Pipeline-Aware Conformal Prediction with Joint Coverage Guarantees for Multi-Stage NLP and LLM Pipelines》,arXiv 2605.18812 —— 链接
• 论文《Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees》,arXiv 2607.04430 —— 链接
• 论文《DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensembles》,arXiv 2603.20975 —— 链接
• 论文《Quantifying Uncertainty of LLM-based Multi-Agent Systems》,ACL 2026 —— 链接
• 文档Future AGI traceAI(Apache 2.0,OpenTelemetry 原生)—— GitHub

• vendorNikhil Pareek, "Evaluating LLM Confidence and Uncertainty (2026): The Calibration Methodology," Future AGI blog, published 2026-02-24 / updated 2026-05-20 — link (primary source for this document's production practices, code examples and ECE figures)
• paper"Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities," arXiv 2602.05073 — link
• paperFarquhar, S. et al., "Detecting hallucinations in large language models using semantic entropy," Nature (2024)
• paperTian, K. et al., "Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback," EMNLP (2023)
• paperPhillips, E., Gustafsson, F. K., Wu, S., Thakur, A., Clifton, D. A., "Entropy Alone is Insufficient for Safe Selective Prediction in LLMs," arXiv 2603.21172, 2026-03-22 — link
• paper"UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making," arXiv 2506.17419 — link
• paper"SAUP: Situation Awareness Uncertainty Propagation on LLM Agent," arXiv 2412.01033 — link
• paper"Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls," arXiv 2607.24343 — link (full-text fetch was rate-limited during research; summarised from search results and flagged as not fully verified)
• paper"PASC: Pipeline-Aware Conformal Prediction with Joint Coverage Guarantees for Multi-Stage NLP and LLM Pipelines," arXiv 2605.18812 — link
• paper"Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees," arXiv 2607.04430 — link
• paper"DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensembles," arXiv 2603.20975 — link
• paper"Quantifying Uncertainty of LLM-based Multi-Agent Systems," ACL 2026 — link
• docsFuture AGI traceAI (Apache 2.0, OpenTelemetry-native) — GitHub