From isolation back to shared containers

OpenAI initially separated large workloads to prevent tenant conflicts, but the growing number of accounts made that model harder to operate. The team moved back to shared containers while adding hot-key throttling, quality-of-service classes and faster server-side partition splitting. The company says the changes have eliminated provisioning and noisy-neighbour incidents for several months. Critical data is replicated across regions, while caches are kept current through change-data-capture streams.

Read speed and the next layer

About 90% of Habitat requests are reads. OpenAI reports that global median read latency fell from roughly 120 milliseconds to 10 milliseconds, a twelvefold improvement; this is the company's own production measurement, not an independent benchmark. The service now supports more than one billion users and over 500 petabytes of data. OpenAI is also developing an in-house, exabyte-scale storage layer based on RocksDB Cloud and Raft, but the published report primarily documents the Habitat and Azure Cosmos DB system operating today.

Sources