130 秒速览

1The 30-second version

一句话:主流 Agent 框架都提供“人工审批 / 取消运行 / 超时”这三个“刹车”,文档暗示一旦暂停、取消或超时,被管控的副作用(发请求、写库、退款)就不会发生。2026-08-08 更新的论文《Stop Means Stop》实测 6 个开源框架,发现这一承诺在运行时没有一个被完整兑现,并给出一个放在 Agent 环境之外的闸门 SoundGate 作为修复。

In one sentence: mainstream agent frameworks ship three “brakes”: human approval, run cancellation and timeouts. Their docs imply that once a run is paused, cancelled or timed out, no gated side effect (an HTTP call, a DB write, a refund) will happen. The paper Stop Means Stop (v3 updated 2026-08-08) tested six open-source frameworks and found that promise is not fully enforced at runtime in any of them, then proposes SoundGate, a gate placed outside the agent environment, as the fix.

先说名词:副作用是指会改变外部世界的动作(调用付款接口、发邮件、删文件),做了就收不回;审批门是“这一步要人点同意才执行”的机制;兄弟分支是同一步里被并行执行的另一个动作。

Quick vocabulary: a side effect is an action that changes the outside world (calling a payment API, sending an email, deleting a file) and cannot be taken back; an approval gate is “this step runs only after a human approves”; a sibling branch is another action executed in parallel within the same step.

数字来源:arXiv 2607.14166 摘要与正文;均为论文作者自测,尚无独立复现。

Numbers: arXiv 2607.14166 abstract and body; all reported by the author, not yet independently reproduced.

2背景:为什么“刹车”会失灵

2Background: why the brakes slip

人在环(human-in-the-loop)是 Agent 安全的标准答案:危险动作前停下来问人。各框架也都给了接口——LangGraph 的 interrupt()、OpenAI Agents SDK 的 needs_approval、LlamaIndex Workflows 的等待事件等。问题在于这些接口大多实现在 Agent 自己的执行循环里:它能暂停“当前这条分支”,却未必能拦住同一时刻其他正在跑的东西。

Human-in-the-loop is the standard answer for agent safety: stop and ask a person before a dangerous action. Frameworks all expose it: LangGraph’s interrupt(), the OpenAI Agents SDK’s needs_approval, wait-for-event in LlamaIndex Workflows, and so on. The catch is that these hooks are mostly implemented inside the agent’s own execution loop: they can pause “this branch”, but not necessarily everything else running at the same moment.

模型越来越喜欢并行工具调用(一步里同时发出多个动作),框架也普遍支持并行节点或并发任务。只要“需要审批的动作”和“另一个有副作用的动作”被排进同一步,暂停就只是半个暂停。

Models increasingly emit parallel tool calls (several actions in one step), and frameworks generally support parallel nodes or concurrent tasks. As soon as “the action needing approval” and “another side-effecting action” land in the same step, the pause is only half a pause.

论文测试的 6 个框架(固定版本):LangGraph 1.2.7(Python)、LlamaIndex Workflows(llama-index-core 0.14.23)、Microsoft Agent Framework(agent-framework-core 1.10.0)、OpenAI Agents SDK 0.17.7、CrewAI 1.15.1(只有事后人工复核,不计入泄漏统计)、LangGraph.js 1.4.7。覆盖四种执行模型、两种语言运行时。

The six frameworks tested (pinned versions): LangGraph 1.2.7 (Python), LlamaIndex Workflows (llama-index-core 0.14.23), Microsoft Agent Framework (agent-framework-core 1.10.0), OpenAI Agents SDK 0.17.7, CrewAI 1.15.1 (post-hoc human review only, excluded from leak counts), and LangGraph.js 1.4.7. They span four execution models and two language runtimes.

3核心概念:四种失灵方式

3Core concepts: four ways the brakes fail

4实验发现

4What the experiments found

论文用三类证据:不依赖模型的差分探针(对比文档承诺与运行时行为)、真实模型驱动未修改的框架、以及13 起公开事故语料做佐证。各框架结果(论文表 2 压缩版,✓ 表示复现出该问题):

The paper uses three kinds of evidence: model-free differential probes (docs vs. runtime behaviour), live models driving unmodified frameworks, and a 13-incident public corpus for corroboration. Per-framework results (condensed from Table 2; ✓ means the failure was reproduced):

5修复思路:SoundGate 外部闸门

5The fix: SoundGate, an external gate

核心洞见:在 Agent 循环内部实现的闸门,会被并行分支绕过。所以把闸门搬到 Agent 执行环境之外,让每个副作用在执行前都必须向它“申请准入”。SoundGate 用 Rust 编写,通过按行分隔的协议通信;每个副作用携带 run ID 和幂等键,状态按 (run, key) 隔离。

The key insight: a gate implemented inside the agent loop can be bypassed by a parallel branch. So move the gate outside the agent’s execution environment and require every side effect to request admission before it executes. SoundGate is written in Rust and spoken to over a line-delimited protocol; each effect carries a run ID and an idempotency key, with state scoped to (run, key).

怎么保证“没有旁路”:对网络出口,论文给出两条内核强制路线——只有回环的网络命名空间(直连外网返回 ENETUNREACH),或在 cgroup 上挂 eBPF 钩子(connect4/6、sendmsg4/6 对非闸门地址返回 EPERM)。较弱的手段包括包装器约定、静态 linter、seccomp,以及用 Landlock 管文件路径。

How “no bypass” is enforced: for network egress the paper offers two kernel-enforced routes: a loopback-only network namespace (direct egress fails with ENETUNREACH), or cgroup eBPF hooks (connect4/6 and sendmsg4/6 return EPERM for any non-gate address). Weaker rungs include wrapper discipline, a static linter, seccomp, and Landlock for filesystem paths.

形式化验证与性能:准入核心用 Verus、TLA+/TLC(最多约 7.5×10⁷ 个状态)、TLAPS 和 Loom 建模验证,并用 1.2×10⁷ 次操作的差分一致性测试比对模型与实际 Rust 代码,零偏差。τ-bench 中每次写入约 1 ms 开销、零误拒;持久化准入约 1.2 万次/秒。据 Developers Digest 报道可 pip install soundgate 安装(本文未核验该包)。

Verification and performance: the admission core is modeled and checked with Verus, TLA+/TLC (up to ~7.5×10⁷ states), TLAPS and Loom, and compared with the deployed Rust code by differential conformance testing over 1.2×10⁷ operations, with zero divergences. In τ-bench it adds about 1 ms per write with zero false refusals, and sustains about 12k durable admissions per second. Developers Digest reports it installs via pip install soundgate (this article did not verify the package).

6怎么用:今天就能做的四件事

6How to use it: four things you can do today

即使不引入 SoundGate,论文的结论也能直接指导代码结构。以下第 1、2 条有框架官方文档支撑,第 3、4 条为作者据论文推导的做法(代码为示意,不是任何库的 API)。

Even without adopting SoundGate, the findings translate directly into code structure. Items 1 and 2 are backed by official framework docs; items 3 and 4 are the author’s practices derived from the paper (code is illustrative, not any library’s API).

① 把副作用放在审批门“下游”,别放在同一步。论文 0/363 的结果说明:位于闸门之后的节点不会泄漏。LangGraph 文档也建议“把副作用放在 interrupt 之后或单独的节点里”。

① Put side effects downstream of the gate, never in the same step. The paper’s 0/363 result shows nodes after the gate do not leak. LangGraph’s docs likewise recommend placing side effects after the interrupt or in a separate node.

from langgraph.types import interrupt

def approve(state):                      # 节点 1:只负责问人,没有副作用
    ok = interrupt({"action": "refund", "amount": state["amount"]})
    return {"approved": ok}

def do_refund(state):                    # 节点 2:在审批之后串行执行
    if state["approved"]:
        refund(state["order_id"],
               idempotency_key=f'{state["run_id"]}:refund:{state["order_id"]}')
    return {}

# 图结构:approve -> do_refund(串行),不要让 do_refund 与 approve 并行

from langgraph.types import interrupt

def approve(state):                      # node 1: only asks the human, no side effects
    ok = interrupt({"action": "refund", "amount": state["amount"]})
    return {"approved": ok}

def do_refund(state):                    # node 2: runs serially after approval
    if state["approved"]:
        refund(state["order_id"],
               idempotency_key=f'{state["run_id"]}:refund:{state["order_id"]}')
    return {}

# graph: approve -> do_refund (serial); never run do_refund in parallel with approve

② 恢复会重跑,所以副作用必须幂等。LangGraph 文档明确:恢复时节点从头执行,“不要在 interrupt 之前做非幂等操作”。OpenAI Agents SDK 的审批按 call ID 记录,可部分批准后再 Runner.run(agent, state) 继续——已决定的调用会继续跑,未决定的再次暂停;这意味着你要清楚哪些调用会在你还没全部处理完时就执行。

② Resume re-runs code, so side effects must be idempotent. LangGraph’s docs are explicit: on resume the node restarts from the top; “do not perform non-idempotent operations before interrupt”. The OpenAI Agents SDK records approvals per call ID and lets you approve some items and then Runner.run(agent, state) again: resolved calls continue while unresolved ones pause the run again, so you need to know which calls will execute before you have handled them all.

③ 在副作用层做“准入”,而不是在 Agent 循环里:哪怕只是一个所有工具共用的包装函数,也能补上 P1–P4 的大部分。

③ Do admission at the side-effect layer, not inside the agent loop: even a single wrapper shared by every tool covers most of P1–P4.

# 示意:所有有副作用的工具都必须经过 admit()
class Gate:
    def __init__(self, store):            # store:持久化 KV(如 Redis/数据库)
        self.s = store
    def admit(self, run_id, key, needs_approval):
        if self.s.get(f"cancel:{run_id}"):            # P4 取消围栏
            raise Refused("run cancelled")
        if self.s.get(f"done:{run_id}:{key}"):        # P3 重放去重
            raise Refused("already executed")
        if needs_approval:
            d = self.s.get(f"decision:{run_id}:{key}")
            if d is None: raise Pending()             # P1 未决定就扣住
            if d == "reject": raise Refused("rejected")  # P2 拒绝即永不放行
        self.s.set(f"done:{run_id}:{key}", 1)         # 注意:真实实现需原子 CAS

# Illustration: every side-effecting tool must call admit()
class Gate:
    def __init__(self, store):            # store: durable KV (e.g. Redis / a database)
        self.s = store
    def admit(self, run_id, key, needs_approval):
        if self.s.get(f"cancel:{run_id}"):            # P4 fence on cancel
            raise Refused("run cancelled")
        if self.s.get(f"done:{run_id}:{key}"):        # P3 dedup on replay
            raise Refused("already executed")
        if needs_approval:
            d = self.s.get(f"decision:{run_id}:{key}")
            if d is None: raise Pending()             # P1 hold until decided
            if d == "reject": raise Refused("rejected")  # P2 rejected stays refused
        self.s.set(f"done:{run_id}:{key}", 1)         # note: a real one needs atomic CAS

④ 让旁路在物理上不可能:包装函数只是“约定”,模型生成的代码或第三方工具仍可能直连外网。生产环境里把 Agent 放进只能访问闸门/代理地址的网络(容器网络策略、命名空间或出口代理),才接近论文说的“完全中介”。

④ Make bypass physically impossible: a wrapper is only a convention; model-written code or third-party tools can still reach the network directly. In production, put the agent on a network that can reach only the gate/proxy (container network policy, a namespace, or an egress proxy) to approach the paper’s “complete mediation”.

7坑与局限

7Pitfalls and limits

论文为单作者预印本(2026-07-15 首次提交,08-08 更新到 v3),尚未见同行评审或框架维护者的公开回应;以上框架版本为论文固定版本,新版本可能已修复部分问题。

The paper is a single-author preprint (first submitted 2026-07-15, v3 on 08-08); no peer review or public response from framework maintainers was found. Framework versions are those pinned by the paper; newer releases may have fixed some issues.

8资源与来源

8Resources and sources