Google TPUs Deploy Production AI at Meta

Google's custom AI chips, known as tensor processing units (TPUs), are now running in production at Meta, a significant step that directly challenges Nvidia's long-held dominance in the AI hardware market. This capability was unlocked by a new PyTorch-native version of the SGLang inference engine, allowing Meta's AI workloads to be deployed on Google's hardware without a complete rewrite of its software.

The announcement, made on Google's developer blog, details a collaboration between Google, Meta, and the SGLang team to enable this deployment. A matching version of another popular inference engine, vLLM, was also released as open source. Previously, both SGLang and vLLM required Google's own JAX AI framework to operate on TPUs, creating a barrier for companies like Meta that build their software on PyTorch. "SGLang-TorchTPU is running in production at Meta today," said Zongwei Zhou, Meta's PyTorch-TPU lead, following what he described as a close three-way collaboration.

For Meta, the ability to run its software on TPUs aligns with a stated strategic goal. "We cannot be constrained by a single hardware architecture," said Jana van Greunen, director of PyTorch engineering at Meta. "TorchTPU enables us to deploy our PyTorch workloads to Google TPUs." Google confirmed it co-developed the TorchTPU backend with Meta's PyTorch team. This is not Meta's first attempt to use TPUs; research teams used an older bridge from PyTorch to TPUs from 2020 to 2023 before Meta canceled those contracts.

The deployment follows a reported multi-year rental agreement for Google Cloud TPU services. While the exact location of Meta's production TPUs is not specified, discussions last year reportedly covered TPU rentals for 2026 with potential purchases in 2027. This move by Meta is part of a broader industry trend. It followed a similar announcement from Amazon, which said it was open-sourcing TorchNeuron, its own native PyTorch backend for its Trainium2 and Trainium3 chips. Both announcements were made at the PyTorch Conference and appear aimed at loosening Nvidia's software lock-in.

Questions about performance have emerged alongside the announcements. In a test run by Inferact, the startup that helped Google build vLLM-TorchTPU, a setup using 16 of Google's seventh-generation Ironwood TPUs delivered over 700 tokens per second for a single user. This test, which used Moonshot AI's 2.8-trillion-parameter Kimi K3 model with speculative decoding, was compared against vLLM's published setup on 16 Nvidia GB200s, which reached 452 tokens per second. The report also states that its kernels for certain models deliver roughly 1.4 to 2 times the GB200's throughput at small batch sizes without speculative decoding.

The SGLang-TorchTPU engine is slated to receive support for additional models, beginning with DeepSeek V4.1 Flash and Reflection Beam. RadixArk, the startup founded by SGLang's creators that built the engine with Google and Meta, stated it is working toward providing support for new open models on their release day by the first quarter of 2027. Google, Meta, and RadixArk plan to provide further details at the SGLang Summit 2026, scheduled for November 12-13 in San Francisco.