Mingyu's Library主页
深度学习文档 · AI Daily(08-03 补生成)Deep-Dive · AI Daily (backfilled 08-03)

视觉 token 压缩
——把文本渲染成图片省 token

Visual Token Compression
Rendering Text as Images to Cut Costs

文本按字符计费,图片按像素计费——把长提示词渲染成一张 PNG,成本立刻换了一套算法。开源工具 pxpipe 用这个反直觉套利把 Claude Code 会话成本砍掉 59–70%,背后连着 DeepSeek-OCR 的「光学压缩」研究方向。本文讲清原理、用法、代价与适用边界。

Text is billed roughly per character; images are billed by pixel dimensions. Render a long prompt as a PNG and the cost follows a different formula. The open-source tool pxpipe uses this counterintuitive arbitrage to cut Claude Code session costs by 59–70%, echoing the "optical compression" research line behind DeepSeek-OCR. This document covers the mechanism, usage, costs, and where it does — and doesn't — fit.

📅 调研日期:2026-08-03📅 Researched: 2026-08-03 🌐 主来源:The Decoder 全文 + 项目仓库信息🌐 Primary: The Decoder full text + project repo details 🀄/🅰️ 中英双语,右上角切换🀄/🅰️ Bilingual — toggle top right

1🔭 开篇速览:30 秒版本🔭 At a Glance: The 30-Second Version

痛点:Agent 会话里真正「新」的内容很少,大头是每轮重复发送的静态大块——系统提示、工具文档、越滚越长的历史。按「约一字符一 token」计费,这些静态内容是持续失血点。

The pain: little in an agent session is actually new; the bulk is static content re-sent every turn — system prompts, tool documentation, ever-growing history. At roughly one token per character, that static mass bleeds money continuously.

解法:pxpipe(7 月初开源,MIT 协议,TypeScript 本地代理)拦截发往 Claude Code 的请求,把静态大块渲染成排版紧密的 PNG,近期消息与模型输出仍走文本。因为图片按像素而非内容计费,每个图像 token 能装下约 3.1 个字符——开发者 Steven Chong 报告平均节省 59–70%,一次 Fable 5 会话从 $42.21 降到 $6.06。本周它持续位居 The Decoder 最热文章第一。

The fix: pxpipe (open-sourced in early July, MIT-licensed, a local TypeScript proxy) intercepts Claude Code requests and renders the bulky static parts as densely packed PNGs, while recent messages and model outputs stay text. Because images are billed by pixels rather than content, each image token carries about 3.1 characters — developer Steven Chong reports 59–70% average savings, with one Fable 5 session dropping from $42.21 to $6.06. It has topped The Decoder's most-read list all week.

Claude Code 请求系统提示 + 工具文档+ 旧历史 + 新消息 pxpipe 本地代理静态大块 → 渲染成紧密 PNG近期消息 → 原样文本直通在你机器上完成,再发出 视觉编码器模型「看图读字」 模型正常作答 实例:约 48,000 字符的系统提示+工具文档 → 文本计费约 25,000 token;渲染成一页 PNG 后约 2,700 token ≈ 每个图像 token 装 3.1 个字符,该部分约 9 倍压缩 Claude Code requestSystem prompt + tool docs+ old history + new messages pxpipe local proxyStatic bulk → dense PNG renderRecent messages → pass as textAll on your machine, then sent Vision encoderModel reads the image Modelanswers normally Example: ~48,000 chars of system prompt + tool docs → ~25,000 tokens as text; ~2,700 tokens as one PNG page ≈ 3.1 characters per image token — roughly 9x compression on that portion
图 1 · pxpipe 管道:静态内容走「像素通道」,动态内容走「文本通道」。数字来自 The Decoder(2026-07-04)。
Fig. 1 · The pxpipe pipeline: static content takes the pixel lane, dynamic content the text lane. Figures from The Decoder (2026-07-04).

2🧮 原理:一场计费套利🧮 The Mechanism: A Pricing Arbitrage

这个技巧成立,靠的是两条计费规则的相遇:文本大约「一字符一 token」;图片则按像素尺寸折算固定 token 数,与图里写了多少字无关。于是把稠密文本(代码、JSON、文档)排版成小字号紧密图片,就等于把内容「塞进」更便宜的计费通道——每个图像 token 约可装 3.1 个字符。

The trick works because two billing rules collide: text costs roughly one token per character, while images cost a fixed token count based on pixel dimensions — regardless of how much text they contain. Typeset dense text (code, JSON, docs) as a tightly packed image and you've smuggled the content into the cheaper billing lane — about 3.1 characters per image token.

💡 深一层:这不只是「钻空子」
💡 One level deeper: not just a loophole

计费差的底层是表示差:视觉编码器天然是种压缩器——一页版面被编码成远少于逐字符的向量序列。「文本的最优输入表示未必是文本」正在从套利技巧变成正经研究方向(见第 4 节)。当然,厂商也可能通过提高图像计费来关掉这扇窗——The Decoder 明确提示了这一风险。

Beneath the billing gap is a representation gap: a vision encoder is inherently a compressor — a page of layout becomes far fewer vectors than character-by-character tokens. "The optimal input representation for text may not be text" is graduating from arbitrage to a legitimate research direction (see §4). Providers could also close the window by repricing images — a risk The Decoder explicitly flags.

3🔧 pxpipe 怎么用🔧 Using pxpipe

1
形态:本地代理,不改客户端 Form factor: a local proxy, no client changes TypeScript 写的本地代理(github.com/teamchong/pxpipe,MIT),坐在 Claude Code 与 API 之间:拦截请求 → 重写静态大块为 PNG → 转发。渲染在本机完成,内容离开机器前已是图片。 A TypeScript local proxy (github.com/teamchong/pxpipe, MIT) sitting between Claude Code and the API: intercept → rewrite static bulk as PNGs → forward. Rendering happens locally; content leaves your machine already as images.
2
压缩对象有讲究:只压「静态且陈旧」的 What gets compressed: only the static and stale 被渲染成图的是系统提示、工具文档、较旧的对话历史;近期消息与模型输出保持文本——模型「正在思考」的工作区不动,压的是「背景资料架」。这个切分是设计的关键。 Rendered to images: system prompts, tool documentation, older chat history; recent messages and model outputs stay text — the model's active workspace is untouched; only the background shelf gets compressed. That split is the key design decision.
3
模型适配是硬约束 Model support is a hard constraint 默认只开 Claude Fable 5 与 GPT 5.6——Fable 5 在「新鲜随机数防背题」的数学基准上读图 100% 准确;而 Opus 4.7/4.8 约 7% 读错、GPT 5.5 也偏弱,均默认关闭、需手动开启。换句话说:这招的可行性取决于具体模型的视觉读字能力,不可盲目迁移。基准细节在仓库 FINDINGS.md。 Only Claude Fable 5 and GPT 5.6 are on by default — Fable 5 reads renders at 100% accuracy on math benchmarks with fresh random numbers it can't have memorized; Opus 4.7/4.8 misread ~7% and GPT 5.5 also underperforms, so both are off by default. In other words: viability depends on each model's visual text-reading ability — don't port it blindly. Benchmark details live in the repo's FINDINGS.md.

4🔬 研究脉络:光学压缩不是新想法🔬 Research Lineage: Optical Compression Isn't New

把文本当图片喂模型,此前最系统的工作是 DeepSeek-OCR:按其技术报告,该系统把文本文档作为图像处理,最高压缩约 10 倍、保留 97% 的信息——目标同样是让模型「装下」更长的文档。pxpipe 可视作这条「光学压缩(optical compression)」路线在计费现实里的民间应用:研究端问「视觉表示能多高效地承载文本信息」,工程端问「这能省多少钱」。两端得到的都是肯定回答,这也是它值得关注的原因——即便计费窗口关闭,表示层的效率红利仍在

The most systematic prior work on feeding text as images is DeepSeek-OCR: per its technical report, the system processes text documents as images with up to ~10x compression while retaining 97% of the information — likewise aiming to fit longer documents into the model. pxpipe is this "optical compression" line applied, folk-style, to billing reality: research asks how efficiently visual representations can carry text; engineering asks how much money that saves. Both get a yes — which is why it matters: even if the billing window closes, the representation-level efficiency dividend remains.

5⚠️ 代价与常见坑⚠️ Trade-offs & Pitfalls

四条必须知道的代价
Four costs you must know

有损:哈希、密钥、精确字符串从图里读回可能出错——涉及 exact-string 的内容绝不能进像素通道。② 更慢:多了渲染 + 视觉编码两道工序,延迟敏感场景不合算。③ 模型依赖:如上节,7% 的读错率(Opus 4.7/4.8)对生产代码是不可接受的;换模型必须重测。④ 窗口风险:如果这类技巧流行,厂商可能上调图像计费,省钱逻辑一夜失效——把它当战术优化,别当架构地基。

Lossy: hashes, keys, and exact strings can come back garbled — never route exact-string content through the pixel lane. ② Slower: rendering plus vision encoding add latency — a poor fit for latency-sensitive paths. ③ Model-dependent: a 7% misread rate (Opus 4.7/4.8) is unacceptable for production code; retest whenever you switch models. ④ Window risk: if the trick spreads, providers may reprice image processing and the savings vanish overnight — treat it as a tactical optimization, not an architectural foundation.

还有一条工程视角的提醒:该方案与提示缓存(prompt caching)存在互动——静态内容本是缓存的最佳对象,若你已用缓存压低了静态部分的重复成本,pxpipe 的边际收益会小于宣传数字;反之,在不支持缓存或历史频繁变动的场景,它的收益最大。上线前先算自己的账。

One more engineering note: the approach interacts with prompt caching — static content is caching's best customer, so if caching already discounts your static repeats, pxpipe's marginal savings will undershoot the headline numbers; conversely it shines where caching is unavailable or history churns. Run your own numbers before adopting.

6🧭 适用判断:一张速查表🧭 When It Fits: A Quick Reference

场景适合吗原因 ScenarioFit?Why
长系统提示 + 大量工具文档的 Agent 会话(Fable 5 / GPT 5.6)✅ 最佳场景静态占比高、模型读图准、成本主导 Agent sessions with long system prompts + heavy tool docs (Fable 5 / GPT 5.6)✅ Best caseHigh static share, accurate visual reading, cost-dominated
涉及哈希/密钥/精确 diff 的流程❌ 禁用有损读回会污染 exact-string Flows involving hashes / keys / exact diffs❌ AvoidLossy read-back corrupts exact strings
延迟敏感的交互式补全❌ 不合算渲染+视觉编码的额外延迟 Latency-sensitive interactive completion❌ Poor fitExtra latency from rendering + vision encoding
已重度使用 prompt caching 的负载🟡 先算账静态部分已被缓存打折,边际收益缩水 Workloads already leaning on prompt caching🟡 Do the mathCaching already discounts the static part; marginal gains shrink

7📚 学习资源清单📚 Learning Resources

报道CoverageThe Decoder 全文(2026-07-04,本文主要事实来源)The Decoder's full article (2026-07-04, primary factual source)
研究脉络Research lineageDeepSeek-OCR 光学压缩报道 · 关联主题:本系列 07-22《上下文退化与自压缩》DeepSeek-OCR optical compression coverage · Related: Context Rot & Self-Compaction (07-22 in this series)

一句话带走

The One-Line Takeaway

视觉 token 压缩 = 计费套利(今天省 60% 的钱)+ 表示革命的预告(文本的最优输入未必是文本)。用它省钱时记住三条铁律:exact-string 不进像素通道、换模型必重测、别把窗口期红利当架构地基

Visual token compression = a billing arbitrage (60% savings today) + a preview of a representation shift (text's optimal input may not be text). While pocketing the savings, keep three iron rules: no exact strings through the pixel lane, retest on every model switch, and never build architecture on a windfall window.