Google and Meta release TorchTPU to run PyTorch natively on TPUs
The open source backend, started at Google in mid 2025 with Meta's PyTorch Team, is already used in production by some of the largest Google Cloud customers, its team said.
AI illustrationGoogle and Meta have introduced TorchTPU, an open source backend that lets PyTorch run natively on Google's TPU chips. The project's team said it began at Google in mid 2025 in collaboration with the PyTorch Team at Meta, and that it is in active development.
According to the team, TorchTPU is already used in production for inference and training workloads by some of the largest Google Cloud customers. TPUs have historically been tied to the TensorFlow and JAX ecosystems, the team noted.
The team described TorchTPU as a break from earlier attempts. Google released PyTorch/XLA, a first compiled version of PyTorch for TPU based on Lazy Tensors, in 2020. The team said that integration needed intrusive changes to source code, suffered from severe compile-time latency, lacked support for PyTorch's eager execution mode and lagged on newer distributed computing features. TorchTPU instead treats eager execution as a top priority and has no JAX dependencies in its runtime.
The stated goal is that moving a workload to a TPU should require little more than switching the target device from a GPU to a TPU in the code. The team cautioned that such a change alone would not deliver peak TPU performance, because a model's shapes, sharding and operations may be tuned for GPUs, but said the model should still run out of the box.
So far, the team said, the project has:
- added support for eager and compiled modes, including a new eager mode called DeferAndFuse;
- added support for about 450 ATen operations;
- enabled custom kernels written through JAX/Pallas and the Helion DSL;
- validated out-of-box execution on the 5,000 most downloaded Hugging Face models;
- validated integration with vLLM, SGLang, TorchTitan, Ray and TorchRec.
The team also listed limits. A TPU device can be accessed by only one user-space process, so reinforcement learning colocation cannot be supported out of the box, and the team said it is investigating ways around this. It is also adding support for bounded dynamism, because the underlying compiler and runtime assume tensor shapes are known at compile time.
On precision, TorchTPU defaults to a medium mode that multiplies in bfloat16 and accumulates in FP32. Higher modes recover more FP32 precision at roughly 3 times or roughly 6 times the compute cost, and the team recommends the default for most deep learning workloads.