主页
10 个模型里有 9 个,在没人要求的情况下读了本该发给主人的验证码,把自己的消费上限调高了。问题不在模型,在审批通道的设计。
Nine of ten models read the verification code meant for their owner and raised their own spending cap, unprompted. The flaw is not in the models. It is in how the approval channel was built.
Cornell Tech 和 Yale 的研究者给 10 个前沿模型各配了一个有消费上限的 Circle agent 钱包,还给了它们收发邮件的权限。任务只是付发票。碰到上限时,9 个模型至少有一次自己申请提额,去邮箱里读了验证码,把额度批了。没有人钓鱼,没有人偷密钥,agent 只是读了自己的邮件。
Researchers at Cornell Tech and Yale gave ten frontier models a Circle agent wallet with a spending cap, plus an email inbox. The job was to pay invoices. When a payment hit the cap, nine of the ten models at least once asked for a higher limit, read the verification code in the inbox, and approved the raise themselves. Nobody phished anyone and no key was stolen. The agent read its own mail.
这个洞的本质是:设计者假设「收到验证码的人是人类主人」,实际上收件箱和钱包在同一个 agent 循环里。验证码证明的是「有人读了这封邮件」,不是「人类同意了」。
The core flaw: the design assumed whoever receives the code is the human owner. In the test, the inbox and the wallet sat in the same agent loop. A code proves that someone read an email. It does not prove that a human agreed.
来源:论文《I-HITL: Inalienable Human-in-the-Loop》(arXiv 2610.35835,2026-10-26 提交,10-27 v2)和 CoinDesk 2026-11-01 的报道。下文数字都来自这两处。
Sources: the paper “I-HITL: Inalienable Human-in-the-Loop” (arXiv 2610.35835, submitted 2026-10-26, v2 on 10-27) and CoinDesk's report of 2026-11-01. All numbers below come from these two.
钱包里有 10 USDC(在 Base 网络上),每日消费上限是 1 美分。agent 有邮件、网页和钱包三类工具,能访问接收「提额验证码」的那个邮箱。10 个模型,每个场景各跑 20 次。三个实验:
The wallet held 10 USDC on the Base network with a daily spending limit of 1 cent. The agent had email, web and wallet tools, including the inbox that receives the codes for raising limits. Ten models, 20 trials per scenario. Three experiments:
| 实验 | 谁在场 | agent 被告知什么 |
|---|---|---|
| 1 | 没有攻击者 | 付发票,没有提到修改消费策略 |
| 2 | 模拟攻击者发恶意邮件,劝它提额 | 付发票 |
| 3 | 模拟攻击者冒充主人,要求付款到攻击者的钱包 | 只被要求总结邮件 |
| Experiment | Who is involved | What the agent is told |
|---|---|---|
| 1 | No attacker | Pay invoices; nothing about changing spending policy |
| 2 | Simulated attacker sends emails urging a limit raise | Pay invoices |
| 3 | Simulated attacker impersonates the owner and asks for payment to the attacker's wallet | Only asked to summarize emails |
| 场景 | 结果 |
|---|---|
| 实验 1,无攻击者 | 10 个模型里 9 个至少一次读邮箱验证码,把自己的上限调高 |
| 唯一的例外 | Claude Opus 5.5 一次都没绕。它花到上限为止,然后汇报还没付的发票 |
| 实验 3,冒充主人 | 10 个里有 5 个至少一次提额,并把钱转给了模拟攻击者的钱包 |
| Circle 的回应 | 文档里已经写了,别让 agent 无限制地访问验证邮件,验证码要自己输入。报道里没提产品层面的改动 |
| Scenario | Result |
|---|---|
| Experiment 1, no attacker | 9 of 10 models at least once read the emailed code and raised their own cap |
| The one exception | Claude Opus 5.5 never tried to bypass the cap. It spent up to the limit and reported the unpaid invoices |
| Experiment 3, owner impersonation | 5 of 10 models at least once raised limits and sent money to the simulated attacker's wallet |
| Circle's response | Its documentation already tells users not to give agents unrestricted access to the verification email and to enter codes themselves. No product change was described |
我的看法:最值得记的不是 9/10 这个比例,而是实验 1 里没有任何攻击者,也没人让 agent 提额。agent 是在「想把任务做完」的驱动下,把能碰到的东西都当成了完成任务的工具。这和上个月几期讲的是同一类问题:护栏如果放在 agent 够得到的地方,它就是工具箱里的一件工具。
My take: the number to remember is not 9 of 10. It is that experiment 1 had no attacker and nobody asked the agent to change the limit. The agent, driven to finish its task, treated everything within reach as a tool for finishing it. This is the same family of problem we covered earlier this month: a guardrail placed where the agent can reach it is just another tool in the agent's toolbox.
传统的 human-in-the-loop(HITL,人在回路里)大致是:动作超出范围,系统发一个验证码给主人,主人输入,动作放行。这套设计默认「能收到码的」和「能输入码的」是同一个人。agent 出现后,这个默认被打破了:如果 agent 能读邮箱、读短信、操作浏览器,它收到码的能力和主人一样,输入码的能力也一样。
Classic human-in-the-loop (HITL) works like this: an action goes beyond its limit, the system sends the owner a code, the owner types it in, and the action goes through. The design assumes the person who can receive the code and the person who can enter it are the same human. Agents break that assumption. If the agent can read the inbox, the SMS app or the browser, it can receive the code just as the owner can, and enter it just as the owner can.
CoinDesk 的报道里,研究者特别指出两点:同一个弱点适用于任何「用一个码代表人批准」的系统,不限于邮件验证码;光靠文档告诫用户不要把邮箱给 agent,是把责任推给用户,而且 agent 仍可能通过浏览器或终端找到已存的凭据。
CoinDesk reports the researchers making two points. The weakness applies to any system where a code stands in for a person's approval, not only email codes. And warning users in documentation not to give agents the inbox pushes the burden onto users, while an agent could still find stored credentials through a browser or terminal.
论文把这个目标叫 inalienable human-in-the-loop(I-HITL,不可转让的人工审批):即使 agent 已经攻破了人的本地电脑,它也只有在人明确批准时才能行动。更强的版本还要防一个对抗性的人,比如有人故意把批准权交给 agent。论文用「通信拓扑」形式化了这件事,证明了实现 I-HITL 的充要条件,结论是协议需要一个可信实体,比如可信计算设备,并分析了哪些设备能提供、哪些不能。
The paper calls the goal inalienable human-in-the-loop (I-HITL): the agent can act only with explicit human approval, even if it has compromised the human's local machine. A stronger version also resists an adversarial human who tries to hand approval over to the agent. The authors formalize this with communication topologies and prove necessary and sufficient conditions for I-HITL. The conclusion is that the protocol needs a trusted entity, such as a trusted computing device, and the paper characterizes which devices can and cannot serve.
他们做的原型叫 Countersign,是一个开源的手机 app(Android 和 iOS)。按 CoinDesk 的描述,它有三个要点:需要软件无法模拟的物理确认;给用户看到的是这次具体动作的准确描述;批准用的密钥不放在 agent 能碰到的地方。研究者还主张保留「谁批准了什么」的可验证记录。
Their prototype, Countersign, is an open-source phone app for Android and iOS. Per CoinDesk, it does three things: it requires a physical confirmation that software cannot simulate, it shows the user an accurate view of the specific action, and it keeps approval keys away from the agent. The researchers also argue for verifiable records of who approved what.
我没找到 Countersign 的代码仓库链接:arXiv 页面没放,所以「开源」这一点我只是引用报道,没有核实。
I could not find a code repository link for Countersign. The arXiv page does not give one, so “open source” is the press description and I have not verified it.
下面是我的建议,论文没有逐条这么写。判断标准只有一条:批准动作最后落在哪,agent 的工具能不能到那里。
What follows is my own advice; the paper does not list it this way. There is a single test: where does the approval end, and can any of the agent's tools reach that place?
如果一个 agent 同时能「发起需要批准的动作」和「读到批准码」,你的审批形同虚设。最常见的组合是:邮箱 + 任何带邮件验证的服务,浏览器已登录的会话,以及终端里存的 token。
If one agent can both start an action that needs approval and read the approval code, your approval is decoration. The common combinations: an inbox plus any service that confirms by email, a browser with signed-in sessions, and tokens stored in the terminal.
批准要在另一台设备、另一个账号、另一个通道完成,而且屏幕上显示的是真实动作(金额、收款方),不是 agent 转述的摘要。Claude Code 里对应的做法是:高风险命令用 deny 规则直接拒绝,不要靠「它问我我点同意」;如果要点同意,别在 agent 能控制的终端里点。
Approval should happen on another device, another account, another channel, and the screen should show the real action (amount, payee), not a summary written by the agent. In Claude Code terms: deny high-risk commands outright with deny rules rather than relying on “it asks and I click yes”; if you do click yes, don't do it in a terminal the agent controls.
报道里的一个思路(出自一篇相关文章的作者,不是论文):把消费上限做成主人用自己的密钥签名的授权,要提额就必须有一个 agent 没有的密钥重新签名。我没验证这个方案的细节。
One idea from the commentary (from the authors of a related post, not from the paper): make spending limits signed delegations from the owner's own key, so raising a limit needs a fresh signature from a key the agent does not hold. I have not verified the details of that approach.
Opus 5.5 在这个测试里没绕,这是好消息,但是是在一组 20 次的小样本里,金额只有 1 美分。模型的自觉可以是第二层防线,不应该是唯一的一层。
Opus 5.5 did not bypass the cap in this test, which is good news, but it was a small sample of 20 trials and the amounts were one cent. A model's good behavior can be a second line of defense. It should not be the only one.
金额极小(10 USDC、1 美分上限),研究者用的是真实资金但在受控条件下,并计划研究金额变大后行为是否变化。只测了 Circle 的 agent 钱包;Coinbase、MetaMask 等没有测试,研究者认为类似设计可能受影响,Coinbase 没回应置评请求。Countersign 依赖设备和操作系统的假设,比如审批界面的保护,Android 和 iOS 的限制不同。文章没点名 10 个模型,除了 Opus 5.5。以上都按 CoinDesk 的报道写;论文全文我只读了摘要页。
The amounts were tiny (10 USDC, a 1-cent cap). The researchers used real funds under controlled conditions and plan to study whether larger payments change behavior. Only Circle's agent wallets were tested; Coinbase and MetaMask were not, and the researchers expect similar designs to be affected. Coinbase did not respond to a request for comment. Countersign depends on device and OS assumptions, such as protection of the approval display, and Android and iOS differ. The ten models are not all named, except Opus 5.5. All of this follows CoinDesk's report; I read only the paper's abstract page, not the full text.