主页
深度学习文档 · 2026-09-02
Deep Dive · 2026-09-02
推理算力已经不再是「买模型自带」的东西,而是一类需要显式分配的资源。effort 调的是每一步想多深,task_budget 调的是整个循环能干多少活——两个旋钮正交,踩坑点却互相纠缠。
Inference compute is no longer something that simply comes with the model — it is a resource you allocate explicitly. effort tunes how deeply Claude reasons at each step; task_budget caps how much total work it does across a loop. The two knobs are orthogonal, but their failure modes are entangled.
2026 年 9 月 1 日,Anthropic 发布 Claude Fable 5.1 与 Mythos 5.1。发布页最值得注意的不是某个基准分数,而是图表的画法:四张性能图全部是「成绩(%)对成本(美元,对数轴)」,每条曲线上标着 low / med / high / xhigh / max 五个点。换句话说,官方不再把模型当成一个固定能力的黑盒来宣传,而是把它当成一条你可以在上面选点的曲线。
On 1 September 2026, Anthropic released Claude Fable 5.1 and Mythos 5.1. The most telling thing on the announcement page is not any single benchmark number — it is how the charts are drawn: all four performance charts plot score (%) against cost (USD, log scale), with five labelled points on each curve: low, med, high, xhigh, max. In other words, the model is no longer marketed as a black box with one fixed capability level. It is marketed as a curve you choose a point on.
同一天还有两件事把这个选择变得更值钱:一是 Fable 5.1 把缓存读取(cache read)单价降了 75%,到 $0.25 / 百万 token,官方称典型负载总成本降约 25%、高度 agentic 负载最多降约 45%;二是 Claude Code v2.1.257 给 /effort 加了「按 s 只改本会话」。前者改变了长会话的成本构成,后者让「按任务选档位」从一句口号变成一个键。
Two things the same day made that choice more valuable. First, Fable 5.1 cut cache-read pricing by 75%, to $0.25 per million tokens; Anthropic reports roughly 25% lower total cost on typical workloads and up to about 45% on highly agentic ones. Second, Claude Code v2.1.257 added a session-only option to /effort (press s). The first changes the cost structure of long sessions; the second turns "pick a level per task" from a slogan into a single keystroke.
effort 控制深度(每一步想多久、调多少工具),task_budget 控制广度(整个 agentic loop 总共能花多少)。两者正交、可叠加,但都会参与 prompt 渲染——所以在依赖缓存的会话中途改动它们,会打掉缓存前缀。正确姿势是按工作负载定值,而不是在一个会话里反复切。
effort controls depth (how long Claude thinks per step, how many tools it reaches for). task_budget controls breadth (how much the whole agentic loop may spend). They are orthogonal and composable — but both participate in the rendered prompt, so changing either mid-session invalidates your cached prefix. Set them per workload, not repeatedly within one conversation.
官方文档对 effort 的定义是「控制 Claude 回应请求时愿意花多少 token」,取值 low / medium / high / xhigh / max,默认 high(显式传 high 与完全不传行为完全一致)。它有两个关键设计:
The docs define effort as controlling "how many tokens Claude spends when responding to requests", with values low / medium / high / xhigh / max and a default of high (passing high explicitly is exactly identical to omitting the parameter). Two design choices matter:
文档明确提醒:effort 是行为信号,不是严格的 token 预算。在低档位下,遇到足够难的问题 Claude 仍然会思考,只是比高档位下想得少。
The docs are explicit that effort is a behavioural signal, not a strict token budget. At lower levels Claude will still think on sufficiently hard problems — just less than it would at a higher level on the same problem.
Task Budgets 目前是 beta(需要 beta header task-budgets-2026-03-13),做法是告诉 Claude 整个 agentic loop 有多少 token 可用——包含 thinking、工具调用、工具结果和输出。服务端在对话中注入一个倒计时标记,模型据此排优先级并优雅收尾。
Task budgets are currently in beta (beta header task-budgets-2026-03-13). You tell Claude how many tokens the full agentic loop has — including thinking, tool calls, tool results, and output. The server injects a countdown marker into the conversation, and the model uses it to prioritise work and finish gracefully.
| 维度 | effort | task_budget |
|---|---|---|
| 控制什么 | 每一步的推理深度与冗长度 | 整个 agentic loop 的总工作量 |
| 作用范围 | 单次请求 | 跨多次请求的一整个循环 |
| 强制性 | 行为信号(非硬限) | 软提示(非硬限),硬限仍是 max_tokens |
| 可见性 | 你设定,响应里不回传 | 只有模型看得见倒计时,响应 usage 里没有 remaining 字段 |
| 状态 | 正式功能,无需 beta header | Beta,需 task-budgets-2026-03-13 |
| 支持面 | API / Bedrock / Vertex / Foundry / Claude Code | 仅 Messages API;Claude Code 与 Cowork 不支持 |
| Dimension | effort | task_budget |
|---|---|---|
| What it controls | Reasoning depth and verbosity per step | Total work across the agentic loop |
| Scope | A single request | A whole loop spanning many requests |
| Enforcement | Behavioural signal (not a hard cap) | Soft hint (not a hard cap); max_tokens remains the hard limit |
| Visibility | You set it; not echoed in the response | Only the model sees the countdown; no remaining field in response usage |
| Status | GA, no beta header needed | Beta, requires task-budgets-2026-03-13 |
| Availability | API / Bedrock / Vertex / Foundry / Claude Code | Messages API only; not supported on Claude Code or Cowork |
| 档位 | 官方描述 | 典型用途 |
|---|---|---|
max | 不设 token 花费约束的绝对最高能力 | 需要最深推理与最彻底分析的任务 |
xhigh | 面向长时程工作的扩展能力 | 超过 30 分钟、token 预算以百万计的长时程 agentic 与编码任务 |
high | 高能力。等价于不设该参数(API 默认) | 复杂推理、困难编码问题、agentic 任务 |
medium | 均衡取舍,适度节省 token | 需要平衡速度、成本与表现的 agentic 任务 |
low | 最高效。显著节省 token,伴随一定能力下降 | 更简单、要求速度与低成本的任务,例如 subagent |
| Level | Official description | Typical use |
|---|---|---|
max | Absolute maximum capability, no constraints on token spending | Tasks needing the deepest reasoning and most thorough analysis |
xhigh | Extended capability for long-horizon work | Long-running agentic and coding tasks (over 30 minutes) with token budgets in the millions |
high | High capability. Equivalent to not setting the parameter (API default) | Complex reasoning, difficult coding problems, agentic tasks |
medium | Balanced approach with moderate token savings | Agentic tasks needing a balance of speed, cost, and performance |
low | Most efficient. Significant token savings with some capability reduction | Simpler tasks needing the best speed and lowest cost, such as subagents |
注意 xhigh 是较新的档位:有些支持 max 的模型并不支持 xhigh。按官方兼容表,xhigh 可用于 Fable 5、Mythos 5、Opus 5、Opus 4.8、Opus 4.7 与 Sonnet 5;max 的支持面更宽,还包括 Opus 4.6、Sonnet 4.6 与 Mythos Preview。
Note that xhigh is a newer level: some models that support max do not support xhigh. Per the compatibility table, xhigh is available on Fable 5, Mythos 5, Opus 5, Opus 4.8, Opus 4.7, and Sonnet 5; max has wider coverage, additionally including Opus 4.6, Sonnet 4.6, and Mythos Preview.
这是最容易被忽略的一点:官方针对不同模型给出的建议起点并不相同,把上一代的 effort 设置直接搬到新模型上是错的。文档明确写着「如果你从更早的模型沿用了 effort 设置,请在你的评测集上重新做一次 effort 扫描,而不是直接复用」。
This is the most commonly missed point: the recommended starting level differs by model, and carrying settings over from a previous generation is a mistake. The docs say plainly: if you carried effort settings over from an earlier model, run a fresh effort sweep on your evals rather than reusing them.
| 模型 | 官方建议起点 | 要点 |
|---|---|---|
| Claude Fable 5 / 5.1 / Mythos 5 | high(默认) | effort 是这代模型「智能 / 延迟 / 成本」的首要控制手段;例行工作大胆降到 medium / low —— 官方称其低档位「常常超过上一代模型 xhigh 的表现」 |
| Claude Opus 5 | high(默认) | 大胆使用 low / medium 作为主要成本控制手段。⚠️ xhigh 与 max 下不能关闭 thinking,传 thinking: {"type":"disabled"} 会返回 400。另外改 effort 不能可靠地缩短可见回答长度,想要短回答请用 prompt 控制 |
| Claude Opus 4.7 / 4.8 | xhigh(编码与 agentic) | 比 4.6 更严格地遵守低档位——低档下会把工作范围收敛到「只做被要求的事」。若在复杂问题上看到推理变浅,提高 effort,而不是靠 prompt 绕 |
| Claude Sonnet 4.6 | medium | 虽然默认是 high,但官方建议显式设置以避免意料之外的延迟 |
| Model | Recommended starting point | Key notes |
|---|---|---|
| Claude Fable 5 / 5.1 / Mythos 5 | high (default) | Effort is the primary control for trading off intelligence, latency, and cost on this generation. Step down to medium/low for routine work freely — the docs note lower settings "often exceed xhigh performance on prior models" |
| Claude Opus 5 | high (default) | Use low/medium liberally as your primary cost and latency control. ⚠️ Thinking cannot be disabled at xhigh or max: thinking: {"type":"disabled"} returns a 400. Also, changing effort does not reliably shorten visible responses — prompt for length instead |
| Claude Opus 4.7 / 4.8 | xhigh for coding and agentic work | Respects lower levels more strictly than 4.6 — at low levels it scopes work to exactly what was asked. If reasoning looks shallow on a hard problem, raise effort rather than prompting around it |
| Claude Sonnet 4.6 | medium | The default is high, but the docs recommend setting effort explicitly to avoid unexpected latency |
在 xhigh 或 max 下运行时,官方反复强调要设一个大的 max_tokens,给模型在 subagent 与工具调用之间思考和行动的空间;从 64k 起步再往下调是个合理的默认值。原因很直接:max_tokens 是「thinking + 回答正文」的总输出硬上限,设小了会在 thinking 阶段就把回答截断。
When running at xhigh or max, the docs repeatedly stress setting a large max_tokens so the model has room to think and act across subagents and tool calls; starting at 64k and tuning from there is a reasonable default. The reason is direct: max_tokens is a hard ceiling on total output — thinking plus response text — so setting it small can truncate the answer during the thinking phase.
这一点在做 Agent 时比「思考深度」更有实感。官方列出的差异是:
When building agents this matters more concretely than "thinking depth". The documented differences:
把多个操作合并进更少的工具调用;总体调用次数更少;不铺垫直接动手;完成后用简短的确认语。
发起更多工具调用;先说明计划再行动;给出详细的变更摘要;写更完整的代码注释。
Combine multiple operations into fewer tool calls; make fewer calls overall; proceed straight to action without preamble; use terse confirmations on completion.
Make more tool calls; explain the plan before acting; give detailed summaries of changes; include more comprehensive code comments.
# Python:请求级设置(注意在 output_config 里,不是顶层参数)
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-fable-5-1",
max_tokens=65536, # xhigh/max 时从 64k 起步
messages=[{"role": "user", "content": "分析微服务与单体架构的取舍"}],
output_config={"effort": "medium"}, # low / medium / high / xhigh / max
)
# Python: a request-level setting (note it lives inside output_config, not at top level)
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-fable-5-1",
max_tokens=65536, # start at 64k for xhigh/max
messages=[{"role": "user", "content": "Analyze the trade-offs between microservices and monoliths"}],
output_config={"effort": "medium"}, # low / medium / high / xhigh / max
)
adaptive 是 thinking 的一种模式,不是 effort 档位。thinking 参数决定 Claude 是否在回答前产出 thinking block;effort 决定整个回应投入多少工作量——在 adaptive 模式下,后者也包含「多频繁、多深地思考」。
adaptive as an effort value
adaptive is a thinking mode, not an effort level. The thinking parameter controls whether Claude produces thinking blocks before answering; effort controls how much work goes into the whole response — which, in adaptive mode, includes how often and how deeply it thinks.
Task Budgets 最反直觉、也最容易用错的一点是:倒计时算的是「Claude 这一轮看到的新内容」,不是你请求体的大小。在 agentic loop 里,你的客户端每轮都会重发整段对话历史,payload 逐轮膨胀;但预算只按这一轮 Claude 新看到的 token 扣减。
The most counter-intuitive — and most misused — property of task budgets is this: the countdown counts what Claude sees this turn, not the size of your request payload. In an agentic loop your client resends the full conversation every turn, so the payload grows turn over turn; the budget only decrements by the tokens Claude newly sees.
max_tokens,达到时以 stop_reason: "max_tokens" 截断。两者相互独立:task_budget 跨整个循环(可能很多次请求),max_tokens 管单次请求,所以并不要求前者小于等于后者。usage 对象里没有 task_budget 信息,SDK 也没有对应访问器。要在客户端追踪花费,只能自己把循环里每次请求的 token 用量加起来。max_tokens, which truncates with stop_reason: "max_tokens". The two are independent: task_budget spans a whole loop (potentially many requests) while max_tokens caps each individual request, so one need not be at or below the other.task_budget information in the usage object and no SDK accessor for it. To track spend client-side, sum token usage across the requests in your loop yourself.# 设置一个任务预算(注意 beta header)
with client.beta.messages.stream(
model="claude-fable-5-1",
max_tokens=128000, # 硬上限:单次请求
output_config={
"effort": "high", # 深度
"task_budget": {"type": "tokens", "total": 64000}, # 广度
},
messages=[{"role": "user", "content": "通读代码库并提出重构方案"}],
betas=["task-budgets-2026-03-13"],
) as stream:
response = stream.get_final_message()
# 跨压缩(compaction)续算:服务端不记得压缩前花了多少,自己传 remaining
tokens_spent_so_far = 45000
output_config = {
"effort": "high",
"task_budget": {
"type": "tokens",
"total": 128000,
"remaining": 128000 - tokens_spent_so_far,
},
}
# ⚠️ 若你每轮都重发完整未压缩历史,就【省略】remaining,让服务端自己算
# Setting a task budget (note the beta header)
with client.beta.messages.stream(
model="claude-fable-5-1",
max_tokens=128000, # hard cap: per request
output_config={
"effort": "high", # depth
"task_budget": {"type": "tokens", "total": 64000}, # breadth
},
messages=[{"role": "user", "content": "Review the codebase and propose a refactor plan."}],
betas=["task-budgets-2026-03-13"],
) as stream:
response = stream.get_final_message()
# Carrying a budget across compaction: the server has no memory of pre-compaction spend
tokens_spent_so_far = 45000
output_config = {
"effort": "high",
"task_budget": {
"type": "tokens",
"total": 128000,
"remaining": 128000 - tokens_spent_so_far,
},
}
# ⚠️ If your loop resends full uncompacted history every turn, OMIT remaining and let the server track it
官方给的方法不是拍脑袋,而是先测量:在不设 task_budget 的情况下跑一批有代表性的任务,记录每个任务的总 token 花费(把循环里每次请求的 usage.output_tokens 加起来,再加上你在请求之间追加的工具结果 token),得到一个分布,从 p99 起步,然后上下调。关键是按你实际的任务长度分布来定,而不是套一个固定默认值。
The documented method is measurement, not guesswork: run a representative sample of tasks without task_budget set and record total tokens per task (sum usage.output_tokens across every request in the loop, plus the tool-result tokens you append between requests). From that distribution, start at the p99 and tune up or down. Size budgets against your actual task-length distribution rather than a fixed default.
Fable 5.1 的定价变动只有一项:缓存读取(cache read)从 $1 降到 $0.25 每百万 token,降幅 75%。输入 $10 / 输出 $50 每百万 token 保持不变。但 Anthropic 给出的实测口径是:典型负载总成本降约 25%,高度 agentic 负载降约 45%(基于 2026 年 8 月四周的真实使用数据,默认 effort)。
Fable 5.1 changed exactly one price: cache reads dropped from $1 to $0.25 per million tokens, a 75% cut. Input at $10 and output at $50 per million tokens are unchanged. Anthropic's measured figures: roughly 25% lower total cost on typical workloads and about 45% on highly agentic ones, based on four weeks of actual August 2026 usage at default effort.
为什么单改一项能拉动这么多总成本?因为在上下文重、工具重的 agentic 负载里,缓存读取就是账单的大头。官方的成本构成图(以 Fable 5 = 100 为指数)是这么分的:
Why does changing one line item move total cost so much? Because in context-heavy, tool-heavy agentic workloads, cache reads are most of the bill. Anthropic's indexed cost chart (Fable 5 = 100) breaks down as:
缓存读取现在便宜了,长会话的经济性完全建立在「前缀稳定、缓存持续命中」之上。而 effort 与 task_budget 都参与 prompt 渲染:改动其中任何一个,新的值都不再匹配旧值创建的缓存条目,整段前缀作废、需要重新写入。
官方给的规则很直接:在依赖缓存命中的长会话里,一开始就选定 effort 并保持不变;把 effort 的变化放在不同工作负载之间,而不是同一段对话内部。
Now that cache reads are cheap, the economics of long sessions rest entirely on a stable prefix that keeps hitting cache. But effort and task_budget both participate in the rendered prompt: change either and the new value no longer matches cache entries created under the old one, so the whole prefix is invalidated and must be re-written.
The documented rule is blunt: pick an effort level at the start of a cache-dependent session and keep it constant; vary effort across workloads, not within one conversation.
Task Budgets 有一个额外的缓存陷阱:倒计时标记是服务端按轮注入的,本身不跨请求匹配缓存。如果你的客户端每轮都自己递减 task_budget.remaining,那个变化的值会让包含它的缓存前缀失效。官方建议:在首次请求上设一次预算,之后让模型对着服务端倒计时自律,不要在客户端不断修改它。
Task budgets add one more caching trap: the countdown marker is injected server-side per turn and does not itself match across requests. If your client decrements task_budget.remaining every turn, that changing value invalidates any cache prefix containing it. The recommendation: set the budget once on the initial request and let the model self-regulate against the server-side countdown rather than mutating it client-side.
把上面的规则拼起来,可以得到一个实际可执行的决策流程。核心思想不是「找到一个最优档位」,而是把不同类型的工作负载分开,各自定档,并在负载内部保持稳定。
Putting the rules together yields a workable decision procedure. The core idea is not to find one optimal level, but to separate workload types, fix a level per type, and hold it constant within each.
Claude Code 侧不支持 task_budget,但 effort 是完整可用的,而且 v2.1.257 之后可以按会话覆盖。Fable 5.1 在 Claude Code 里默认 high(在 Cowork 与 claude.ai 上默认 medium)。
Claude Code does not support task budgets, but effort is fully available, and since v2.1.257 it can be overridden per session. Fable 5.1 defaults to high effort in Claude Code (and to medium in Cowork and on claude.ai).
# 会话内:打开选择器,选好档位后按 s → 只对当前会话生效
/effort
# 启动时指定,只影响这一次会话
claude --effort low -p "把 README 的安装步骤同步成最新的 pnpm 命令"
# 常备别名:把「杂活会话」和「硬骨头会话」分开
alias ccq='claude --model claude-fable-5-1 --effort low'
alias ccd='claude --model claude-fable-5-1 --effort xhigh'
# 顺带把 subagent 一律压到低档模型(v2.1.257 新增 FORCE)
CLAUDE_CODE_SUBAGENT_MODEL=claude-haiku-4-5-20251001 \
CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 claude
# In session: open the picker, then press s to apply to this session only
/effort
# At launch, for this session only
claude --effort low -p "Update the README install steps to the current pnpm commands"
# Standing aliases: separate the "chores" session from the "hard problem" session
alias ccq='claude --model claude-fable-5-1 --effort low'
alias ccd='claude --model claude-fable-5-1 --effort xhigh'
# And force every subagent onto a cheaper model (FORCE added in v2.1.257)
CLAUDE_CODE_SUBAGENT_MODEL=claude-haiku-4-5-20251001 \
CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 claude
不要凭感觉。在 Claude Code 里用 /cost 看每会话的输出 token 与 prompt cache 明细(命中率、miss 次数、重新缓存的 token 数);在 API 侧把循环里每次请求的 usage.output_tokens 加起来。判据是:降档之后输出 token 明显下降,而你在自己的评测集上没看到质量下降。官方对所有降档建议都加了同一个前提——「只在你已经测量到低档位能保住质量之后才降」。
Don't go by feel. In Claude Code, read /cost for per-session output tokens and the prompt-cache line (hit ratio, misses, tokens re-cached). On the API, sum usage.output_tokens across the loop. The test is: output tokens drop visibly after stepping down, and your evals show no quality regression. Every step-down recommendation in the docs carries the same precondition — step down only once you have measured that the lower level holds quality on your evals.
| 症状 | 可能原因 | 怎么办 |
|---|---|---|
| 长会话里缓存命中率突然崩掉 | 会话中途改了 effort 或 task_budget —— 两者都参与 prompt 渲染 | 在会话开始时定好并保持不变;把变化放在不同工作负载之间 |
| 设了 task_budget 之后出现莫名拒绝或提前收尾 | 预算相对任务明显偏小,模型选择不接或缩小范围 | 先调大预算,再去 debug 其他参数;按实测 p99 定值,最小 20,000 |
| xhigh / max 下回答被截断 | max_tokens 太小,thinking 就把额度吃完了 | 从 64k 起步设 max_tokens,再向下调 |
Opus 5 上 thinking: disabled 报 400 | Opus 5 在 xhigh / max 下不允许关闭 thinking | 降到 high 及以下,或不要关 thinking |
| 调低 effort 但回答还是很长 | Opus 5 上 effort 控制的是思考量,不可靠地控制可见长度 | 用 prompt 明确要求长度,而不是靠 effort |
| 复杂问题上推理明显变浅 | Opus 4.7/4.8 在低档位下严格收敛工作范围 | 提高 effort,而不是用 prompt 绕;必须保持低延迟时,加一句「这个任务涉及多步推理,回答前请仔细思考」 |
| 换新模型后成本/质量都变了样 | 把上一代的 effort 设置直接沿用了 | 在自己的评测集上重新做一次 effort 扫描 |
| 倒计时掉得比预期快、模型早早收尾 | 重发完整历史的同时,客户端还在递减 remaining | 重发完整未压缩历史时省略 remaining,让服务端算;只有做了 compaction 才传 |
| Claude Code 里找不到 task_budget | Task budgets 不支持 Claude Code 与 Cowork | 只能通过 Messages API 使用;Claude Code 侧用 effort + 上下文管理替代 |
| gateway 会话里选不到 Fable 5.1 | 别名 fable / best 暂时仍解析到 Fable 5 | 在 /model 里显式选 Fable 5.1 |
| Symptom | Likely cause | What to do |
|---|---|---|
| Cache hit rate collapses mid-session | effort or task_budget changed mid-conversation — both participate in the rendered prompt | Fix the value at session start and hold it; vary across workloads instead |
| Unexplained refusals or early stops after setting a task budget | The budget is clearly too small for the task, so the model declines or scopes down | Raise the budget first, then debug other parameters; size from measured p99, minimum 20,000 |
| Truncated answers at xhigh / max | max_tokens too small — thinking consumes the allowance | Start max_tokens at 64k and tune down |
400 error on Opus 5 with thinking: disabled | Opus 5 cannot disable thinking at xhigh or max | Drop to high or below, or leave thinking on |
| Lower effort but responses are still long | On Opus 5, effort controls thinking volume, not visible length | Prompt explicitly for length instead of relying on effort |
| Reasoning looks shallow on hard problems | Opus 4.7/4.8 scope work tightly at lower levels | Raise effort rather than prompting around it; if latency forces low effort, add "This task involves multistep reasoning. Think carefully before responding." |
| Cost and quality both shifted after a model upgrade | Effort settings were carried over from the previous generation | Run a fresh effort sweep on your own evals |
| Countdown drops faster than expected; model wraps up early | Client decrements remaining while also resending full history | Omit remaining when resending full uncompacted history; pass it only after compaction |
| Can't find task_budget in Claude Code | Task budgets are not supported on Claude Code or Cowork | Use them via the Messages API; in Claude Code, rely on effort plus context management |
| Can't select Fable 5.1 in a gateway session | The aliases fable and best still resolve to Fable 5 for now | Pick Fable 5.1 explicitly in /model |
厂商暴露 effort 这个旋钮,背后是一条已经比较成熟的研究线:自适应 test-time compute 分配。它的出发点是一个经验观察——投入算力与准确率增益之间是凹函数关系(边际收益递减)。既然如此,与其对所有请求均匀投入,不如把算力挪到边际收益最高的地方。
The reason vendors expose an effort knob at all traces back to a fairly mature research line: adaptive test-time compute allocation. It starts from an empirical observation — the relationship between added compute and marginal accuracy gain is concave (diminishing returns). Given that, spending uniformly across all requests is wasteful; the win comes from reallocating compute to where marginal return is highest.
目前的主流做法大致分三类:
Current approaches fall broadly into three families:
用显式或代理的复杂度度量估计一个查询有多难,据此分配资源——这正是 effort 这类参数在产品层的对应物。
用 PRM 或外部 verifier 的信号,自适应地剪枝、扩展或停止推理轨迹。
监测潜空间或输出空间的收敛情况,达到条件就提前停止,省下剩余算力。
Estimate query complexity with explicit or proxy measures and allocate resources accordingly — the research counterpart of a product-level effort parameter.
Use PRM or external verifier signals to adaptively prune, expand, or halt reasoning trajectories.
Monitor convergence in latent or output space and stop early once the criterion is met, saving the remaining compute.
代表性工作如 Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization(arXiv:2604.14853)。另有独立测量指出:在 LLM API 上,选一个运行时长两倍的模型,大致能把错误率降低同一个倍数,且这个关系与初始运行时长无关——这为「用延迟换准确率」提供了一个粗略的定价直觉。⚠️ 后一条来自第三方分析(Epoch AI),不是厂商数据,引用前请回溯原文。
A representative paper is Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization (arXiv:2604.14853). Separately, independent measurement suggests that across LLM APIs, choosing a model with roughly 2× longer runtime cuts error by about the same factor, largely independent of the initial runtime — a rough pricing intuition for trading latency against accuracy. ⚠️ That second finding comes from third-party analysis (Epoch AI), not vendor data; check the original before citing.
对工程实践的启示很直接:推理算力应该被当作一类需要策略性分配的一等资源,而不是均匀花掉。产品侧的 effort + task_budget,本质上就是把这条研究结论做成了两个可以直接调的参数——难度感知路由由你(或你的路由层)来做,自我调速由模型对着倒计时来做。
The engineering takeaway is direct: inference compute should be treated as a first-class resource allocated strategically, not spent uniformly. Product-side, effort plus task_budget is essentially that research conclusion turned into two dials you can set — difficulty-aware routing is yours (or your router's) to do, while self-pacing is the model's, against the countdown.
| 术语 | 解释 |
|---|---|
| effort(努力档位) | 请求级参数,位于 output_config 内,五档 low/medium/high/xhigh/max。控制 Claude 在这次回应上愿意花多少 token,作用于全部输出 token。 |
| task budget(任务预算) | Beta 参数,给整个 agentic loop 一个 token 总额。模型能看到服务端注入的倒计时,据此排优先级并收尾。软提示,非硬限。 |
| agentic loop(智能体循环) | 模型「思考 → 调工具 → 读工具结果 → 再思考」的多轮往返,直到给出最终回答等待人类输入为止。 |
| thinking(思考块) | 模型在正式回答前产出的推理内容。thinking 参数决定是否产出;effort 决定在 adaptive 模式下想多频繁、多深。 |
| adaptive thinking(自适应思考) | 一种 thinking 模式,由模型自行决定何时思考、思考多少。它是 thinking 的取值,不是 effort 的取值。 |
| prompt caching(提示缓存) | 把已处理过的提示前缀缓存下来,后续请求命中即可按更低单价读取。Fable 5.1 上缓存读取为 $0.25/Mtok。 |
| cache read(缓存读取) | 模型读取此前已处理并缓存的输入。上下文重的 agentic 负载里,它常常是账单的最大项。 |
| compaction(压缩) | 在长对话中把早期轮次总结/重写以控制上下文长度。做了 compaction 之后需要用 remaining 手动续算任务预算。 |
| max_tokens | 单次请求生成 token 的硬上限(thinking + 正文合计),达到时 stop_reason: "max_tokens"。 |
| PRM(过程奖励模型) | Process Reward Model,对推理过程的中间步骤打分的模型,常用于引导轨迹剪枝或提前停止。 |
| Term | Meaning |
|---|---|
| effort | Request-level parameter inside output_config, with five levels: low/medium/high/xhigh/max. Controls how many tokens Claude spends on this response; affects all output tokens. |
| task budget | Beta parameter giving the whole agentic loop a total token allowance. The model sees a server-injected countdown and uses it to prioritise and wrap up. Advisory, not enforced. |
| agentic loop | The multi-turn cycle of think → call tool → read result → think again, until the model produces a final answer and awaits human input. |
| thinking | Reasoning content the model produces before its answer. The thinking parameter decides whether blocks are produced; effort decides how often and how deeply in adaptive mode. |
| adaptive thinking | A thinking mode where the model decides when and how much to think. It is a value of thinking, not of effort. |
| prompt caching | Caching an already-processed prompt prefix so later requests can read it at a lower price. Cache reads on Fable 5.1 cost $0.25/Mtok. |
| cache read | The model reading input it has already processed and stored. In context-heavy agentic workloads this is often the largest line on the bill. |
| compaction | Summarising or rewriting earlier turns of a long conversation to control context length. After compaction you must carry the task budget forward manually via remaining. |
| max_tokens | The hard per-request ceiling on generated tokens (thinking plus response text); hitting it yields stop_reason: "max_tokens". |
| PRM | Process Reward Model — a model that scores intermediate reasoning steps, often used to guide trajectory pruning or early stopping. |
/effort 会话级覆盖、--effort 语义变更、Fable 5.1 默认模型、gateway 别名说明/effort, the --effort semantics change, Fable 5.1 as default model, gateway alias note