主页
Prompt 缓存不是一个「打开就省钱」的开关,而是一门需要主动设计请求结构的手艺。厂商宣传的是「最高 90%」,跨三家 500+ 真实 agent 会话的实测区间是 41–80%,而多数团队的起点是 7%。这份文档讲清缓存到底怎么命中、怎么被打破、以及六步把命中率提上去。
Prompt caching is not a switch you flip to save money — it's a discipline that requires designing the shape of your requests. Vendors advertise "up to 90%." A measurement across 500+ real agent sessions on three providers found 41–80% in practice. Most teams start at 7%. This guide covers how a cache hit actually happens, what breaks it, and six steps to fix your hit rate.
如果你只读一段,读这段。
If you read only one paragraph, read this one.
Prompt 缓存(prompt caching)的原理很朴素:模型把你请求开头那一段一模一样、一字不差的内容的计算结果存起来,下次同样的开头直接复用,只算新增的部分。省钱和省延迟都来自这里。关键词是前缀(prefix)——它只从头开始匹配,遇到第一个不同的 token 就停止,后面全部重算。
Prompt caching works on a plain idea: the model stores the computed state for the opening stretch of your request — byte-for-byte identical — and reuses it next time, computing only what's new. Both the cost savings and the latency savings come from that. The operative word is prefix: matching starts at position zero and stops at the first token that differs. Everything after that point is recomputed.
这就带来了整件事最反直觉的一点:缓存是否命中,取决于你把动态内容放在哪里,而不取决于动态内容有多少。一个 4,000 token 的系统提示里塞了一个每次都变的会话 ID,如果它在中间,那 4,000 token 每次全额重算;把它挪到末尾,前面 3,900 多个 token 全部命中。有实测记录的一个安全情报 Agent,做的正是这一个改动:命中率 7% → 74%,月度推理账单降 59%。(来源 [2])
Which leads to the least intuitive fact in the whole topic: whether you get a cache hit depends on where the dynamic content sits, not on how much of it there is. A 4,000-token system instruction with one per-request session ID buried in the middle recomputes all 4,000 tokens every time. Move that ID to the end and roughly 3,900 of them hit cache. A documented security-intelligence agent made exactly that one change: hit rate went 7% → 74% and the monthly inference bill dropped 59%. (Source [2])
打开你的用量面板,看 cache read token 占总输入 token 的比例。如果这个数低于 50%,而你的 Agent 每轮都发同一份系统提示和工具定义,那你几乎肯定在为已经算过的东西重复付钱。
Open your usage dashboard and look at cache-read tokens as a share of total input tokens. If that number is under 50% while your agent sends the same system prompt and tool definitions every turn, you are almost certainly paying twice for work the model already did.
聊天应用里缓存是锦上添花;在 Agent 循环里它是成本结构的主导项。
In a chat app, caching is a nice-to-have. In an agent loop, it dominates the cost structure.
单轮问答的成本模型很简单:发一份 prompt,收一份回答。Agent 不是这样。一个 Agent 完成一个任务要跑十几轮甚至上百轮,每一轮都要把完整的历史重新发一遍——系统提示、工具定义、之前所有的思考与工具结果。第 20 轮发送的输入,可能是第 1 轮的十几倍。
Single-turn Q&A has a simple cost model: send a prompt, get an answer. Agents don't work that way. Completing one task takes a dozen turns, sometimes a hundred, and every turn resends the entire history — system prompt, tool definitions, all prior reasoning and tool results. The input on turn 20 can be an order of magnitude larger than on turn 1.
这正是缓存的理想场景:每一轮的输入都是上一轮输入的严格前缀延长(append-only)。理论上,第 N 轮只需要为「第 N−1 轮之后新增的那点内容」付全价,前面全部按 0.1× 计费。理论上。
That is the ideal case for caching: each turn's input is a strict append-only extension of the previous turn's. In theory, turn N pays full price only for what was added since turn N−1, and everything before it bills at 0.1×. In theory.
把模型想成一个每天早上都要重读一遍项目背景才能开工的同事。缓存等于让他「昨天读到哪、今天从哪接着读」。但他有个怪癖:只要你在文档开头改了一个字——哪怕只是把日期从 8 月 20 日改成 8 月 21 日——他就会从第一页重新读起。
Think of the model as a colleague who has to re-read the whole project background every morning before starting work. Caching means he picks up where he stopped yesterday. But he has a quirk: change one character near the front of the document — even just the date, from Aug 20 to Aug 21 — and he starts over from page one.
Agent 循环把这个怪癖放大了。因为历史很长,前缀失效的惩罚不是「多算一点」,而是「把已经积累的整个上下文全额重算」。一个 100k token 的 Agent 会话,前缀被破坏一次,这一轮就要按全价重付 100k 输入 token,而不是 0.1× 的 10k 等效价。这就是为什么一份实践分析把它称作「Agent 循环打破缓存并抽干你的预算」。(来源 [8])
The agent loop amplifies that quirk. Because the history is long, a prefix miss doesn't mean "a bit of extra compute" — it means recomputing the entire accumulated context at full price. In a 100k-token agent session, one broken prefix costs you 100k input tokens at full rate rather than the 10k-equivalent you'd have paid at 0.1×. Hence the framing in one production analysis: agent loops break caching and drain your budget. (Source [8])
三个概念:前缀匹配、断点、TTL。理解这三个,后面全是推论。
Three concepts: prefix matching, breakpoints, TTL. Everything else follows from them.
前缀匹配(prefix matching)
缓存查找只从请求的第 0 个 token 开始,逐 token 比对,遇到第一个不同就停。停下来的位置之前算命中,之后全部重算。它不做「中间那段一样」的模糊匹配——不是搜索,是精确的开头比对。
Prefix matching
Cache lookup starts at token 0 of the request and compares token by token until the first difference. Everything before that point is a hit; everything after is recomputed. There is no fuzzy "the middle section is the same" matching — it isn't search, it's an exact comparison from the front.
缓存断点(cache breakpoint)
你在请求里显式标记「到这里为止的内容请存下来」的位置。Anthropic 用 cache_control 参数标记,每个请求最多 4 个断点。OpenAI 走的是自动前缀缓存,不需要你标记。
Cache breakpoint
An explicit marker saying "store everything up to here." Anthropic marks these with the cache_control parameter, with up to 4 breakpoints per request. OpenAI uses automatic prefix caching instead, requiring no markers from you.
TTL(存活时间)
缓存条目多久过期。Anthropic 的每个断点有 ephemeral TTL,默认 5 分钟(每次命中刷新),也可选 1 小时——1 小时档的写入价是基础输入价的 2×。同一请求里两种 TTL 可以混用,但 1 小时的条目必须排在 5 分钟的前面。
TTL (time to live)
How long an entry survives. Each Anthropic breakpoint carries an ephemeral TTL: 5 minutes by default, refreshed on every hit, or 1 hour as an option — the 1-hour tier writes at 2× base input price. You can mix both TTLs in one request, but 1-hour entries must appear before 5-minute ones.
图 1 · 前缀缓存只从头匹配。一个 12 token 的动态字段放在中间,就让它后面的 3,800 个静态 token 全部作废;挪到末尾,损失降到 12 个 token。这不是优化技巧,这是缓存机制的直接推论。
Figure 1 · Prefix caching matches from position zero. A 12-token dynamic field in the middle invalidates the 3,800 static tokens behind it; moved to the end, the loss shrinks to 12 tokens. This isn't a clever trick — it falls straight out of how the mechanism works.
手动 vs 自动、折扣多少、什么时候值得。
Manual vs. automatic, how big the discount is, and when it pays off.
| 维度 | Anthropic Claude | OpenAI | Google Gemini |
|---|---|---|---|
| 触发方式 | 自动缓存 + 可选 cache_control 显式断点(每请求最多 4 个) | 自动前缀缓存,零代码改动 | 隐式 + 显式两种 |
| 缓存读取折扣 | 0.1× 基础输入价(约省 90%) | 约省 50% | 约 90% 折扣(2.5 系列) |
| 缓存写入成本 | 1.25× 基础输入价(5 分钟档);1 小时档 2× | 不额外收写入费 | 另有存储计费维度 |
| TTL | 5 分钟(命中即刷新)或 1 小时 | 由平台管理 | 可配置 |
| 最小可缓存长度 | 按模型不同,如 1,024 / 2,048 token 起 | 有最小长度门槛 | 有最小长度门槛 |
| Dimension | Anthropic Claude | OpenAI | Google Gemini |
|---|---|---|---|
| How it triggers | Automatic caching plus optional explicit cache_control breakpoints (max 4 per request) | Automatic prefix caching, zero code changes | Both implicit and explicit modes |
| Cache read discount | 0.1× base input price (~90% off) | Roughly 50% savings | ~90% discount on 2.5-series models |
| Cache write cost | 1.25× base input (5-min tier); 2× for the 1-hour tier | No separate write charge | Separate storage billing dimension |
| TTL | 5 minutes (refreshed on hit) or 1 hour | Platform-managed | Configurable |
| Minimum cacheable length | Model-dependent, e.g. from 1,024 / 2,048 tokens | Has a minimum-length threshold | Has a minimum-length threshold |
来源 [1][3][4]。定价与门槛数字随模型代际变化,落地前请以各家当前官方文档为准。
Sources [1][3][4]. Pricing and thresholds shift between model generations; check each vendor's current docs before you build on these numbers.
因为 1 小时档的写入价是 2×(而 5 分钟档是 1.25×),你需要在这一小时内至少读 3 次才能比 5 分钟档划算;5 分钟档只需要读 2 次就回本。实践建议是:交互式、流量稳定 → 用默认 5 分钟;流量突发或请求之间间隔长 → 考虑 1 小时,批处理是典型场景(一个批次跑完往往超过 5 分钟)。如果流量尖峰且不可预测,5 分钟是更安全的默认值。(来源 [3])
Because the 1-hour tier writes at 2× (against 1.25× for the 5-minute tier), you need at least three reads within that hour to come out ahead; the 5-minute tier breaks even at two reads. The practical guidance: interactive workloads with steady traffic → stay on the 5-minute default; bursty or widely spaced requests → consider the hour, with batch processing as the obvious case (a batch often takes longer than five minutes to work through). If your traffic is spiky and unpredictable, 5 minutes is the safer default. (Source [3])
官方建议是从自动缓存开始——它以最小成本覆盖了大多数用例;只有当你确实需要细粒度控制(比如想把工具定义和系统提示分成两个独立的缓存段,让工具集变化时不牵连系统提示)时,才切到显式断点。过早手动标断点是常见的过度工程。
The documented recommendation is to start with automatic caching — it covers the majority of use cases with minimal effort. Switch to explicit breakpoints only when you genuinely need fine-grained control, e.g. splitting tool definitions and system prompt into separate cache segments so a change to the tool set doesn't invalidate the prompt. Reaching for manual breakpoints too early is a classic case of over-engineering.
按顺序做,前两步通常就能拿走大部分收益。
Do them in order. The first two usually capture most of the win.
图 2 · 一个可以照抄的请求排布。核心规则只有一条:按「变化频率」从低到高排,让最稳定的内容占据最长的前缀。Anthropic 每请求最多 4 个断点,把它们放在变化频率的边界上。
Figure 2 · A layout you can copy. One rule drives it: sort by change frequency, lowest first, so the most stable content occupies the longest possible prefix. Anthropic allows four breakpoints per request — place them on the change-frequency boundaries.
# ❌ 反模式:动态内容混在静态内容中间
system = f"""你是一个代码审查助手。
当前时间:{datetime.now()} # ← 每次都变,毁掉后面全部
会话:{session_id} # ← 同上
{3000_tokens_of_review_guidelines} # ← 静态,但已经缓存不到了
"""
# ✅ 正确:静态在前,动态压到最后
system = [
{
"type": "text",
"text": REVIEW_GUIDELINES, # 3000 token,永不变
"cache_control": {"type": "ephemeral", "ttl": "1h"},
},
{
"type": "text",
"text": TOOL_USAGE_NOTES, # 变化频率:发版时
"cache_control": {"type": "ephemeral"}, # 默认 5 分钟
},
{
"type": "text",
"text": f"当前时间:{now}\n会话:{session_id}", # 动态,断点之后
},
]
# 注意:1 小时 TTL 的段必须排在 5 分钟 TTL 的段之前# ❌ Anti-pattern: dynamic content buried among static content
system = f"""You are a code review assistant.
Current time: {datetime.now()} # ← changes every call, voids everything below
Session: {session_id} # ← same problem
{3000_tokens_of_review_guidelines} # ← static, but now uncacheable
"""
# ✅ Correct: static first, dynamic pushed to the tail
system = [
{
"type": "text",
"text": REVIEW_GUIDELINES, # 3,000 tokens, never changes
"cache_control": {"type": "ephemeral", "ttl": "1h"},
},
{
"type": "text",
"text": TOOL_USAGE_NOTES, # changes only on release
"cache_control": {"type": "ephemeral"}, # 5-minute default
},
{
"type": "text",
"text": f"Current time: {now}\nSession: {session_id}", # dynamic, after the breakpoint
},
]
# Note: 1-hour TTL segments must come before 5-minute TTL segments写法依据 Anthropic 官方 prompt caching 文档 [1];具体字段名与可选值请以当前文档为准。
Structure follows Anthropic's prompt caching documentation [1]; verify field names and accepted values against the current docs.
宣传值、实测值、和你大概率会拿到的值。
The advertised number, the measured number, and the one you'll probably get.
先说结论:各家宣传的「最高 90%」是理论上限,不是你的预期值。一份 2026 年 1 月的研究在 OpenAI、Anthropic 与 Google 上测量了 500+ 个真实 agent 会话,得出的实际节省区间是 41–80%。这个差距不是厂商在夸大——90% 指的是「缓存读 vs 全价输入」的单价折扣,而 41–80% 是「一整个会话的实际账单差异」,中间隔着写入成本、未命中的部分和 output token。(来源 [2])
The headline first: the "up to 90%" every lab markets is a theoretical ceiling, not your expected value. A January 2026 study measured caching across 500+ real agent sessions on OpenAI, Anthropic and Google and found actual savings of 41–80%. That gap isn't vendors exaggerating — 90% describes the unit-price discount on a cache read versus full-price input, while 41–80% describes the difference in a whole session's bill, with write costs, unhit segments and output tokens sitting in between. (Source [2])
延迟收益比成本收益小但真实:多份实践报告给出的首 token 时间(TTFT)缩短 13–31%。对交互式产品,这个数量级足以被用户感知到。(来源 [2][6])
The latency win is smaller than the cost win but real: practice reports converge on a 13–31% reduction in time-to-first-token. For interactive products that magnitude is perceptible to users. (Sources [2][6])
2026 年被反复提到的「成本优化三件套」是:为静态结构做前缀缓存、为增长中的上下文做对话压缩、把语义缓存或计划缓存作为上层优化。这三层解决的是不同的问题,不能互相替代——前缀缓存省的是重复计算,压缩省的是上下文体积,语义缓存省的是「同样的问题被问了第二遍」。(来源 [5])
The stack most often described for 2026 cost optimisation has three layers: prefix caching for static structure, conversation compression for a growing context window, and semantic or plan caching as an optimisation layer on top. They solve different problems and don't substitute for each other — prefix caching saves repeated computation, compression saves context volume, semantic caching saves you from answering the same question twice. (Source [5])
写缓存比普通输入 token 贵 25%(Anthropic 5 分钟档)。如果一个前缀只被用一次,开缓存是净亏损。回本点在同一前缀被读 2–3 次。一次性的批量分类、单轮的分类接口、每次 prompt 都不同的场景,不要开。
A cache write costs 25% more than an ordinary input token (Anthropic's 5-minute tier). If a prefix is used only once, caching is a net loss. Break-even is 2–3 reads of the same prefix. One-shot bulk classification, single-turn classifier endpoints, and workloads where every prompt differs should leave it off.
学术侧也在跟进:arXiv 上有一篇《Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks》专门评测长时程 agent 任务下的缓存行为,标题本身就点明了这个领域的核心矛盾。(来源 [7])
Academia is catching up too: an arXiv paper titled Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks evaluates cache behaviour specifically under long-horizon agent workloads — the title alone names the central tension in this area. (Source [7])
这些都是有据可查、反复出现的失败模式,不是假想的。
Documented, recurring failure modes — not hypotheticals.
很多 agent 框架会在运行时把时间戳、用户 ID、会话配置直接注入系统提示。系统提示里差一个字符,整个前缀缓存就作废,每一步都被迫全量重算。这是最隐蔽的一类,因为你的业务代码看起来完全正确。定位方法:抓两次相邻请求的完整 payload,做逐字节 diff。
Many agent frameworks inject timestamps, user IDs, or session configuration straight into the system prompt at runtime. A single character of difference in the system prompt invalidates the prefix cache entirely, forcing a full recompute at every step. This is the sneakiest failure because your own code looks perfectly correct. How to find it: capture the full payload of two consecutive requests and diff them byte for byte.
有一个公开记录的 bug 类型是:自动注入的上下文每 N 轮重建一次系统提示,在任何做前缀缓存的后端上都会导致 KV 缓存整体失效。这类问题的共同特征是「周期性的账单尖峰」——每 N 轮就有一轮特别贵。(来源 [8])
One publicly documented bug class: auto-injected context rebuilds the cached system prompt every N turns, invalidating the KV cache on every prefix-caching backend. The signature of this family of problems is a periodic billing spike — one turn in every N costs far more than its neighbours. (Source [8])
如果一个工具抓的是股价或天气,而你把包含这些结果的完整上下文一起缓存了,后续请求会基于过期数据推理。这条比其他几条更严重,因为它是正确性 bug 而不是成本 bug——账单看起来很漂亮,答案却是错的。规则:缓存边界必须停在任何实时工具输出之前,而不是之后。
If a tool fetches stock prices or weather and you cache the full context including those results, later requests reason over stale data. This one is more serious than the others because it's a correctness bug rather than a cost bug — the bill looks great and the answers are wrong. The rule: the cache boundary belongs before any live tool output, never after.
动态函数调用会在工具定义变化时打破缓存。工具定义包含在 prompt 里,可用工具集的任何变化都会作废前缀。常见诱因包括:工具顺序被随机化、MCP 频繁重连导致工具列表重建、按上下文动态增删工具。修法是把工具列表排序固定、把动态工具装载移到缓存边界之后。
Dynamic function calling breaks the cache whenever tool definitions change. Tool definitions live inside the prompt, so any change to the available tool set invalidates the prefix. Common causes: randomised tool ordering, frequent MCP reconnection rebuilding the tool list, and context-dependent addition or removal of tools. The fix is a deterministic tool ordering plus moving dynamic tool loading past the cache boundary.
这一条在 2026-08-20 有了一个具体的公开例证:Claude Code v2.1.237 修复了「使用 LLM gateway 或自定义 base URL 的会话中 prompt caching 失效」的问题。中间层是缓存最容易悄无声息失效的地方——它可能重写 header、规范化 payload、或者干脆不透传 cache_control。如果你在自建网关后面跑 Agent,把缓存命中率作为网关的一项回归测试。(来源 [9])
This one got a concrete public example on 2026-08-20: Claude Code v2.1.237 shipped a fix for prompt caching being broken in sessions that go through an LLM gateway or a custom base URL. Middle layers are where caching most often dies silently — they may rewrite headers, normalise payloads, or simply fail to pass cache_control through. If you run agents behind your own gateway, make cache hit rate a regression test on the gateway itself. (Source [9])
图 3 · 缓存覆盖长度随轮次增长,直到某个操作重写了历史开头。压缩、时间戳注入、工具集变化都会造成同一个形状的断崖。评估「要不要压缩」时,别只算省下的上下文,要减去缓存重建的成本。
Figure 3 · Cached prefix length grows turn over turn until something rewrites the head of the conversation. Compaction, timestamp injection and tool-set changes all produce the same cliff. When deciding whether to compact, subtract the cost of rebuilding the cache from the context you save.
诚实的边界:这不是万能优化。
The honest boundary: this is not a universal optimisation.
缓存命中率工程之所以值得单独当成一件事来做,是因为它不需要牺牲任何东西:不降低模型能力、不减少上下文、不改变输出质量。它唯一要求的是你对「请求里每一段内容变化多快」有一个清晰的认识,然后按这个认识排序。在所有 LLM 成本优化手段里,这大概是唯一一个纯粹的免费午餐。
Cache hit-rate engineering earns its own name because it costs you nothing in return: no reduction in model capability, no context removed, no change in output quality. All it asks is that you know how fast each part of your request changes, and then order them accordingly. Among LLM cost optimisations, it's about the only genuinely free lunch on the menu.
标记说明:官方=厂商文档或发布说明;二手=第三方实践文章、聚合站或媒体报道,数字可能未经独立复核。
Tags: Official = vendor documentation or release notes; Secondary = third-party practice write-ups, aggregators or press, whose figures are not independently verified here.