Mingyu's Library主页
AI Daily · 2026-10-13 · 深度学习AI Daily · 2026-10-13 · Deep dive

Agent 规模下的 Git 基础设施:GitHub 为什么要把存储和计算拆开

Git Infrastructure at Agent Scale: why GitHub is splitting storage from compute

基于 GitHub 工程博客(2026-10-06)及多方报道整理。读完你会知道:agent 为什么压垮 Git、新旧架构差在哪、该怎么借鉴。

Based on GitHub’s engineering blog (2026-10-06) and press coverage. You will learn why agents strain Git, what differs between the old and new designs, and what to borrow.

130 秒速览

1The 30-second version

一句话:AI 智能体让“提交代码”变得又多又快,GitHub 发现原来的 Git 存储架构被写入拖慢了,于是把“存数据”和“处理请求”拆开——数据只存一份权威版本在对象存储里,处理请求的机器变成可随时增减的缓存,只有“分支指向哪个提交”这一件事需要大家达成一致。

In one sentence: AI agents commit far more often and far faster, and GitHub found its old Git storage was being throttled by writes. So it split “storing data” from “serving requests”: one authoritative copy lives in object storage, request-serving machines become disposable caches, and only “which commit does this branch point to” needs agreement.

先说名词:Git 是记录代码每次修改历史的工具;push 是把本地提交上传到服务器;分支引用(ref)是一个“名字 → 提交”的指针,比如 main 现在指向哪次提交。

Quick vocabulary: Git records the history of every code change; a push uploads local commits to a server; a ref is a “name → commit” pointer, e.g. which commit main currently points to.

旧:Spokes 全量副本新:存储与计算分离一次 push文件服务器 1(全量副本)文件服务器 2(全量副本)文件服务器 3(全量副本)每次写入:三阶段提交 + 法定人数加读容量 = 加一份完整副本↳ push 速度 ≈ 最慢的那个副本一次 pushAzure Blob(唯一权威数据)计算 worker 1(缓存)计算 worker 2(缓存)计算 worker 3(缓存)仅分支引用更新需要协调↳ 读容量独立扩展;worker 挂了 = 缓存未命中
图 1 · 旧架构 vs 新架构(据 GitHub 博客描述绘制的示意图)
Old: Spokes full replicasNew: storage / compute splitOne pushFile server 1 (full copy)File server 2 (full copy)File server 3 (full copy)Every write: 3-phase commit + quorumMore read capacity = one more full replica↳ a push is as slow as the slowest replicaOne pushAzure Blob (source of truth)Compute worker 1 (cache)Compute worker 2 (cache)Compute worker 3 (cache)Only the branch-ref update needs agreement↳ reads scale independently; a lost worker = a cache miss
Figure 1 · Old vs new architecture (sketch based on GitHub’s description)
为什么值得读:同样的问题(很多 agent、很多小提交、同一个分支被反复拉取)你在自己的 CI、monorepo、agent 流水线里迟早遇到,只是规模小几个数量级。GitHub 和 Cursor 的解法给出了一个通用思路:只对必须一致的东西做共识,其余并行或后台做。
Why read this: the same pressures (many agents, many small commits, one branch fetched over and over) will hit your own CI, monorepo or agent pipeline, just several orders of magnitude smaller. GitHub’s and Cursor’s answers share a general idea: reach agreement only on what must be consistent; do everything else in parallel or in the background.

2背景:为什么 agent 会“压垮”Git

2Background: why agents strain Git

以前的写入主要来自人:一天提交几次,中间有大量思考时间。智能体不一样——它们在紧凑的循环里工作,几乎每做一步动作就提交或检查点一次,于是“单次 push 要多久”直接决定 agent 能跑多快。GitHub 在博客里归纳了五个压力点:

Writes used to come mostly from people: a few commits a day with plenty of thinking time between them. Agents work in tight loops and commit or checkpoint after nearly every action, so “how long one push takes” directly caps how fast an agent can go. GitHub’s post lists five pressure points:

  1. 单 agent 提交延迟:一次 push 的耗时就是 agent 的速度上限。
  2. 写吞吐:同一仓库里成千上万个 agent 各自开分支,持续写入汇聚到架构的同一个点。
  3. 引用争用(ref contention):主干开发、发布列车、合并队列把合并都挤向同一个分支指针。
  4. 读扇出:一次 push 会引发成千上万次读取——CI 和代码扫描每分钟对同一个分支末端反复 clone/fetch。
  5. 维护成本:压缩与垃圾回收能保持仓库高效,但成本随写入量增长。
  1. Per-agent commit latency: one push’s duration is the agent’s speed limit.
  2. Write throughput: thousands of agents on their own branches in one repo produce sustained writes that converge on a single point in the architecture.
  3. Ref contention: trunk-based development, release trains and merge queues funnel merges into one ref.
  4. Read fan-out: one push becomes thousands of reads, as CI and code scanning clone/fetch the same branch tip thousands of times a minute.
  5. Maintenance cost: compaction and garbage collection keep repos fast, but their cost grows with write volume.
218.2B → 473.3B月度 Git 事件
2025-09 → 2026-08
7.38B2026-09 提交数
约去年同期 5 倍以上
0.69B → 3.35B月度 push 数
同比 4.9×
3.26B2026-09 Actions 运行
同比 4 倍以上
218.2B → 473.3Bmonthly Git events
Sep 2025 → Aug 2026
7.38Bcommits, Sep 2026
5×+ year over year
0.69B → 3.35Bpushes per month
4.9× YoY
3.26BActions runs, Sep 2026
4×+ YoY

数据来源:GitHub 工程博客;Techzine 补充:中位数仓库每月约 25 次请求,最忙的仓库在 8 月约 10 亿次。The Register 另报道 GitHub 在 4 月和 5 月分别出现 10 次和 9 次性能劣化事件(该数字来自媒体,非 GitHub 博客)。

Data: GitHub engineering blog. Techzine adds that the median repository gets about 25 requests a month while the busiest handled roughly one billion in August. The Register separately reports 10 performance-degrading incidents in April and 9 in May (a media figure, not from GitHub’s post).

3核心概念

3Core concepts

Spokes
GitHub 现有的 Git 存储系统(前身为 2016 年介绍的 DGit):每个仓库默认在 5 台文件服务器的本地磁盘上各存一份完整副本。
三阶段提交 + 法定人数(quorum)
一种让多台机器“一起决定”写入是否生效的协议;“法定人数”指必须有足够多(通常是多数)副本确认。通俗说:开会表决,没凑够人就不能通过。
引用(ref)
名字到提交的指针(如 main)。更新引用才决定“谁看到的是最新代码”,所以它是唯一必须一致的点。
计算 / 存储分离
“保管数据的仓库”和“处理请求的柜台”分开部署。仓库只有一份权威版本,柜台可随时增减。
压缩(compaction)与垃圾回收(GC)
Git 把对象打包整理、清掉无用对象,让仓库保持快;写得越多,要整理的越多。
写前日志(WAL)
先把“要做什么”顺序记到持久日志里再生效,崩溃后可据此重建状态(Cursor 的方案用到它)。
Spokes
GitHub’s current Git storage system (successor to DGit, introduced in 2016): by default each repository has a full copy on the local disks of five file servers.
Three-phase commit + quorum
A protocol by which several machines jointly decide whether a write takes effect; “quorum” means enough replicas (typically a majority) must confirm. Think of a vote that fails if too few people show up.
Ref
A name-to-commit pointer (e.g. main). Updating a ref decides who sees the “latest” code, which is why it is the one thing that must be consistent.
Storage / compute separation
The “warehouse that keeps the data” and the “counters that serve requests” are deployed separately: one authoritative copy, counters you can add or remove at will.
Compaction & garbage collection (GC)
Git packs objects and discards unused ones to stay fast; the more you write, the more there is to tidy.
Write-ahead log (WAL)
Record “what will be done” in a durable, ordered log before applying it, so state can be rebuilt after a crash (used in Cursor’s design).

4机制:旧架构为什么卡,新架构怎么改

4Mechanism: why the old design stalled and what changed

旧架构的根因(GitHub 博客):磁盘上的副本既是持久的权威数据,又是对外答复请求的那一层。想多扛读请求,就得多加一份持久副本;而每个副本都参与每一次写入,所以一次 push 只能和最慢的副本一样快;副本丢失会减少读容量,失去法定人数则整个写入停摆。博客称这套设计目前支撑约 10 亿个仓库。

Root cause of the old design (GitHub’s post): the on-disk copies are both the durable source of truth and the layer that answers requests. To serve more reads you add another durable replica; but every replica takes part in every write, so a push is only as fast as the slowest one; losing a replica reduces read capacity, and losing quorum halts writes entirely. The post says this design serves about a billion repositories today.

新架构的三个动作:

Three moves in the new design:

  1. 最小化协调:只有引用更新需要达成一致;存对象、校验连通性、密钥扫描可与其他写入并行,缩短 push 的关键路径。
  2. 存储与计算分离:权威数据放在 Azure Blob Storage(由它负责持久与复制);轻量计算 worker 缓存数据并响应 Git 请求。worker 丢了就像缓存未命中,换一台立刻接手并从持久存储回填——旧设计里则要重建一整份仓库副本。
  3. 维护移出服务路径:压缩与 GC 交给独立 worker,直接在持久存储上做,忙碌仓库可在后台优化而不拖慢 push/fetch。
  1. Minimize coordination: only the ref update needs agreement; storing objects, validating connectivity and secret scanning can run in parallel with other writes, shortening the push critical path.
  2. Decouple storage from compute: authoritative data lives in Azure Blob Storage (which provides durability and replication); lightweight compute workers cache data and serve Git requests. Losing a worker behaves like a cache miss: a replacement serves traffic at once and refills from durable storage, whereas the old design meant rebuilding a full repository copy.
  3. Move maintenance off the serving path: compaction and GC run on separate workers directly against durable storage, so busy repos are optimized in the background without slowing pushes or fetches.
可并行(不需要全体同意)写入对象数据到持久存储校验对象连通性密钥扫描引用更新 = 唯一需要共识决定谁看到“最新”后台 worker:压缩(compaction)/ 垃圾回收(GC)——移出服务路径来源:GitHub 工程博客(2026-10-06);图为作者据其描述绘制的示意
图 2 · 新设计下一次 push 的关键路径(作者据博客描述绘制的示意)
Can run in parallel (no global agreement)Write object data to durable storageValidate object connectivitySecret scanningRef update = the only consensus stepdecides who sees “latest”Background workers: compaction / garbage collection — off the serving pathSource: GitHub engineering blog (2026-10-06); sketch drawn by the author from its description
Figure 2 · The critical path of a push in the new design (author’s sketch from the post)

效果与护栏:GitHub 称内部基准写吞吐最高提升 35 倍,且读容量不再与写互相拖累。但博客没有给出基线配置和负载,所以 35× 只能当作“方向性数字”。GitHub 同时强调:重构在服务在线时进行、无需维护窗口、用户无需改动;分支保护、必需评审、审计日志、仓库可见性保持不变。

Result and guardrails: GitHub reports up to 35× higher write throughput in internal benchmarks, with read capacity no longer fighting writes. But the post does not give the baseline configuration or workload, so treat 35× as directional. GitHub also stresses that the rebuild happens while the service stays live, with no maintenance window and no user changes; branch protections, required reviews, audit logs and repository visibility are preserved.

5同行对比:Cursor 的 S3 写前日志方案

5Peer comparison: Cursor’s S3 write-ahead-log design

不止 GitHub。据 InfoQ(2026-09-30)报道,Cursor 的 “Continuity” 把对象存储当作事实来源:pushed 数据写入 S3,每次引用更新记录到 S3 支持的 WAL;本地 NVMe 只是热缓存,副本之间不互相协调,而是靠对象存储的原子 compare-and-swap 与条件读取来验证。The Register 也提到 Cursor 以对象存储 WAL + SSD 缓存取代了三阶段提交。

GitHub is not alone. According to InfoQ (2026-09-30), Cursor’s “Continuity” treats object storage as the source of truth: pushed data goes to S3 and each ref update is recorded in an S3-backed WAL; local NVMe is only a warm cache, replicas do not coordinate with each other but validate through atomic compare-and-swap and conditional reads on object storage. The Register also notes Cursor replaced three-phase commit with an object-storage WAL plus SSD caching.

旧 Spokes(GitHub)新 GitHub 方向Cursor Continuity
权威数据多台文件服务器本地盘Azure BlobS3 + WAL
读扩展加持久副本加计算 worker加缓存副本(自测至 100 个线性)
需要共识的每次写入(3PC)仅引用更新S3 原子 CAS
维护在服务路径附近独立后台 worker仅主节点压缩,副本下载 pack
公开数字—写吞吐最高 35×(内部基准)S3 Standard ≈120 push/s;Express One Zone >300 push/s(合成测试,厂商自报)
Old Spokes (GitHub)New GitHub directionCursor Continuity
Source of truthLocal disks on several file serversAzure BlobS3 + WAL
Read scalingAdd durable replicasAdd compute workersAdd cache replicas (linear to 100 in their tests)
What needs consensusEvery write (3PC)Ref updates onlyAtomic CAS on S3
MaintenanceNear the serving pathSeparate background workersPrimary-only compaction; replicas download packs
Public numbers—Up to 35× writes (internal benchmark)~120 pushes/s on S3 Standard; >300 on Express One Zone (synthetic, vendor-reported)
可信度提示:Cursor 的数字来自其自有 everysphere monorepo 的合成测试;InfoQ 引用 Liatrio 的 Casey Lee 指出这些性能数字尚未独立验证。GitHub 的 35× 同样是内部基准。两者都是“厂商自报”,选型时请自行压测。
Confidence note: Cursor’s figures come from synthetic tests on its own monorepo; InfoQ cites Liatrio’s Casey Lee noting they have not been independently verified. GitHub’s 35× is also an internal benchmark. Both are vendor-reported, so benchmark for yourself before choosing.

6对你意味着什么(场景与做法)

6What it means for you (scenarios and practice)

下面是作者基于上述架构原理的推断,不是 GitHub 官方建议:

The following is the author’s inference from the architecture, not official GitHub guidance:

  • agent 流水线:既然“每步一提交”会把 push 延迟变成速度上限,可考虑把检查点合并(例如每完成一个小任务提交一次),或把中间状态放在本地/临时存储。
  • CI:成千上万次对同一分支末端的 clone/fetch 是读扇出的来源;用浅克隆、缓存或镜像可减轻压力(GitHub 博客指出读相对容易靠缓存与副本扩展)。
  • 合并队列/主干开发:引用争用意味着“大家都往 main 合”本身是瓶颈,批量合并或分层分支能缓解。
  • 自建 Git 服务:若你的写入也在增长,可参考“对象存储做权威 + 本地盘做缓存 + 只对引用做原子更新”的思路,但务必先评估强制推送、时间点恢复等一致性问题(InfoQ 引述的开放问题)。
  • Agent pipelines: if “a commit per step” turns push latency into a speed cap, consider batching checkpoints (e.g. one commit per finished sub-task) or keeping intermediate state in local/ephemeral storage.
  • CI: thousands of clones/fetches of the same branch tip are the source of read fan-out; shallow clones, caches or mirrors can ease it (GitHub notes reads are comparatively easy to scale with caches and replicas).
  • Merge queues / trunk-based dev: ref contention means “everyone merges into main” is itself a bottleneck; batched merges or layered branches can help.
  • Self-hosted Git: if your writes are growing too, the pattern “object storage as truth + local disks as cache + atomic updates only for refs” is worth studying, but first evaluate force-push semantics and point-in-time recovery (open questions raised in InfoQ’s coverage).

行业上下文(The Register 报道):前 GitHub CEO Thomas Dohmke 的 Entire 服务会把 agent 流量分流到镜像仓库——说明“把 agent 读写和人类工作流隔开”也是一条并行的思路。

Industry context (The Register): former GitHub CEO Thomas Dohmke’s Entire service offloads agent traffic to mirror repositories, another way to separate agent load from human workflows.

7坑与未解问题

7Pitfalls and open questions

官方明确没讲的:GitHub 博客只是系列概览,没有给出上线时间表、新一致性模型的完整细节,也没说引用更新与加锁在新设计里如何实现;作者表示后续文章会深入。35× 的基线与负载同样未公开。
What GitHub explicitly leaves out: the post is an overview of a series and gives no rollout timeline, no full detail of the new consistency model, and nothing on how locking and ref updates are implemented; the author says follow-up posts will go deeper. The baseline and workload behind 35× are not disclosed either.
社区与媒体视角:The Register 评论称 GitHub 需要“奇迹般”的扩容才能跟上需求,同时也指出工程师面对的是难以预料的需求;PlanetScale CEO Sam Lambert 说多数公司为 10%–20% 的流量增长都要准备数月。这些是评论观点,不是技术事实。
Press and community view: The Register argues GitHub will need a near-miracle of scaling to keep up, while noting the engineers faced demand nobody foresaw; PlanetScale CEO Sam Lambert says most companies spend months preparing for 10–20% traffic growth. These are opinions, not technical facts.
读者常见误解:“存储和计算分离”不等于“不要一致性”。恰恰相反——它把一致性的代价集中到最小的一点(分支引用),其余交给对象存储的持久与复制能力。
A common misreading: “separating storage from compute” does not mean “no consistency”. It concentrates the cost of consistency in the smallest possible place (the branch ref) and leaves the rest to the durability and replication of object storage.

8资源与来源

8Resources and sources