From OpenAI · · 2m
Scaling Git at OpenAI
OpenAI's new Git system handles 1 billion users with faster pushes and reliable merges.
In brief
OpenAI's new Git system handles 1 billion users with faster pushes and reliable merges. OpenAI launched Pando, a Git system that scales to handle 1 billion users with 5x faster pushes and 99% cache efficiency. Originally reported by OpenAI.
Why OpenAI needed new Git system
OpenAI's monorepo faced slow fetches and failed merges as Codex increased parallel development demands.
Pando's dual goals
Pando scales writes across branches and maximizes read throughput for 1 billion users.
Pando's causal consistency model
Pando uses causal consistency to allow parallel writes without full repository consistency.
Pando vs. existing Git systems
Pando outperforms three-phase commit and write-ahead log systems in high-concurrency environments.
Pando's packfile reuse
Pando reuses packfiles across clients to reduce network bandwidth and server load.
Pando's metadata store
Pando stores ref versions in a horizontally scalable metadata service for efficient packfile selection.
Pando's compaction system
Pando uses geometric compaction to reduce fetch latency and improve packfile efficiency.
Pando's deployment timeline
Pando moved from development to production in 10 weeks for OpenAI's monorepo.
Pando's performance metrics
Pando cut median push latency by 5x and delivers nearly a petabyte of git data per day.
Pando's cache efficiency
Pando achieves 99% cache hit rates with regional caches for low-latency git operations.
Pando's contribution to Git
OpenAI contributed improvements to git's packfile-uri and explicit haves features.
Pando's future plans
OpenAI will share Pando technology with other organizations for agent-scale development.
Pando's impact on Codex
Codex helped developers migrate to Pando with skills for checkout and branch management.
Pando's scalability
Pando handles 57 terabytes of git packs in one hour with minimal server resources.
Pando's real-world testing
Pando reduced median push latency by 80% and improved merge reliability in OpenAI's monorepo.
Pando's next steps
OpenAI will share Pando's technology with other organizations for agent-scale development.
The takeaway
OpenAI launched Pando, a Git system that scales to handle 1 billion users with 5x faster pushes and 99% cache efficiency.
