# OpenAI scales storage for 1 billion ChatGPT users

> OpenAI uses Azure Cosmos DB to handle 20M QPS for 1 billion ChatGPT users.
- Title: OpenAI scales storage for 1 billion ChatGPT users · OpenAI
- Summary: OpenAI uses Azure Cosmos DB to handle 20M QPS for 1 billion ChatGPT users. OpenAI scaled Azure Cosmos DB to serve 20 million queries per second for over 1…
- Keywords: azure cosmos db, openai, chatgpt, habitat, nfl, storage, qps, 20m, 1 billion users, azure
- Source: OpenAI — https://openai.com/index/scaling-storage-one-billion-users-part-two
- Read time: 1 min
- Topics: azure cosmos db, openai, chatgpt, habitat, nfl, storage, qps, 20m, 1 billion users, azure, ai, technology
## Habitat platform
OpenAI's online storage platform serves over 1 billion users across 40 geographic regions.
## Scaling challenge
Habitat must respond in milliseconds to 1 billion users across 40 regions.
## Phase I: Azure Cosmos DB
Initially used one default Azure Cosmos DB container per account.
## Logical partition key limits
Collisions on user_id or workspace_id caused throughput constraints.
## Noisy neighbors
One tenant consuming resources disrupted core login and signup flows.
## Weak partitions
Physical partitions split as data grew, reducing effective throughput per partition.
## Phase II: Single-tenancy
Created separate containers for critical use cases to isolate workloads.
## Lack of stable growth
New services required 1,000x more headroom than legacy systems.
## Capacity planning issues
Product teams forgot to pre-scale containers, causing launch delays.
## Phase III: Multi-tenancy
Implemented co-managed containers with hot key throttling and qos.
## Hot key throttling
Throttled hottest keys first to prevent noisy neighbor effects.
## Quality of Service
Assigned lower priority to offline workloads to prevent throttling.
## Serverless mode
Improved serverless mode allowed 10k ru/s per physical partition.
## Reliability improvements
Joint engineering with Azure reduced outages for 10s of petabytes of data.
## Global performance
Reduced global read p50 from 120 ms to 10 ms through caching and rust.
## Future plans
Building custom exabyte-scale storage for next billion users.
## Key takeaway

OpenAI scaled Azure Cosmos DB to serve 20 million queries per second for over 1 billion ChatGPT users.