We run one of the largest Kubernetes clusters in the world. Having one massive cluster lets us iterate rapidly at small scale, and also scale up large models without changing code. Tremendous effort by our supercomputing team (who is hiring: openai.com/jobs/!).