TPU vs GPU on OCI: Choosing the Right AI Accelerator for Each Workload
Once Google TPUs reach Oracle Cloud Infrastructure in 2027, OCI customers will be able to choose between three families of AI accelerator in one cloud. The right answer is rarely one chip for everything. Here is how to decide, workload by workload.
On this page
- 01 · One Cloud, Three Kinds of AI Accelerator
- 02 · What Is on OCI's AI Compute Menu
- 03 · How TPUs and GPUs Differ in Design
- 04 · The Software Question Comes First
- 05 · Matching Workloads to Accelerators
- 06 · Timing: Planning Around a 2027 Arrival
- 07 · A Practical Evaluation Plan
- 08 · Key Takeaways
- 09 · Frequently Asked Questions
01 · One Cloud, Three Kinds of AI Accelerator
Decision guide · AI accelerators on OCI
"Should we wait for TPUs, or keep buying GPUs?"
That question started landing in architecture reviews the day after Oracle AI World. On 27 October 2026 Oracle and Google announced that Google TPUs will be offered through OCI, the first time Google has made its TPUs available on a hyperscale cloud outside its own. OCI already runs NVIDIA GPU clusters at very large scale and AMD Instinct GPU instances. From 2027, a single OCI tenancy could in principle hold all three.
More choice is good news, but it moves a decision onto your desk that used to be made for you. If your only option was a GPU, you only had to pick a size. Now you have to decide which accelerator family suits each workload, how portable your code is between them, and whether a 2027 arrival date fits your roadmap. This post gives you a practical way to make that call without guessing at details Oracle has not published yet.
02 · What Is on OCI's AI Compute Menu
Answer: OCI offers NVIDIA GPU instances and OCI Supercluster for large-scale training, AMD Instinct MI355X instances that are generally available today, and a planned AMD MI450 supercluster. Google TPUs join the portfolio in calendar 2027. All of them sit next to the same OCI storage, database and networking services.
Oracle's own framing of the TPU deal is about matching workloads to infrastructure, not replacing anything. The release says TPUs give customers another option alongside existing accelerators, on infrastructure "designed for near-linear scalability and highly optimized economics". OCI provides the power, cooling and datacenter networking that host the TPUs; Google provides the chips and the software stack around them.
The figure below shows how the three accelerator families could sit side by side in one OCI region. Your tenancy submits training jobs and serves inference endpoints. A placement layer, which today is your own scheduler, quotas and team conventions, routes each job to the cluster that fits it. All three clusters share the region's networking and its data services: Object Storage for datasets and checkpoints, File Storage for shared working sets, and Oracle AI Database 26ai with AI Vector Search for retrieval.
The point of the picture is that data gravity stays the same whichever chip you choose. Your datasets, checkpoints and vector stores live in OCI services, so moving a job from a GPU cluster to a TPU pod is a compute decision, not a data migration. How OCI will actually expose and schedule TPU capacity is still unannounced.
| Accelerator family on OCI | Status (late 2026) | Software stack | Typical OCI use today |
|---|---|---|---|
| NVIDIA GPUs | Available; OCI Supercluster advertised up to 131,072 GPUs for some generations | CUDA and the wider NVIDIA ecosystem | Frontier training, fine-tuning, inference of every size |
| AMD Instinct GPUs | MI355X instances GA; MI450 supercluster of 50,000 GPUs planned | ROCm, PyTorch, JAX, vLLM | Memory-heavy inference, PyTorch training |
| Google TPUs | Announced for calendar 2027, ramping over several years | JAX, PyTorch via TorchTPU, XLA, vLLM | None yet on OCI; Google lists training and inference workloads |
03 · How TPUs and GPUs Differ in Design
Answer: A GPU is a general-purpose parallel processor with a huge, mature software ecosystem. A TPU is an application-specific chip built around large matrix-multiply units, compiled through XLA and designed to scale as tightly connected pods. GPUs win on flexibility; TPUs aim to win on scale and cost per unit of work for the model types they target.
GPUs started as graphics processors and became the default for AI because they run almost any parallel code. Thousands of cores, high-bandwidth memory and a programming model (CUDA on NVIDIA, ROCm and HIP on AMD) let researchers write custom kernels for new ideas the week they are published. That flexibility is why nearly every new model architecture appears on GPUs first.
TPUs were designed by Google for one job: the dense linear algebra at the heart of neural networks. You do not program them kernel by kernel. Your framework hands a computation graph to the XLA compiler, which maps it onto the chip and across the pod. Google builds the pod as the unit of scale, with dedicated chip-to-chip links and, in the eighth generation, separate designs for training (TPU 8t) and inference (TPU 8i). Google says a TPU 8t superpod reaches up to 9,600 chips with about 2 PB of shared HBM.
The comparison below puts the three families next to each other on the five questions that usually decide a placement: what software they run, how portable your existing code is, how they scale, what they are best at, and when you can actually use them on OCI.
Read the bottom row first. Today only the two GPU families can run production work on OCI, so every TPU cell above it is a planning input, not a deployment option. For 2027 and beyond, the portability row matters most, because it decides how much engineering effort a move between families will cost you.
04 · The Software Question Comes First
Answer: Before comparing chips, inventory your code. Framework-level models in JAX, or PyTorch that compiles cleanly, can move between accelerators with modest effort. Custom CUDA kernels and CUDA-only libraries cannot run on TPUs at all and must be rewritten or replaced.
The TPU software story improved sharply during 2026. JAX remains TPU native. TorchTPU, Google's eager-first PyTorch runtime for TPU built with Meta, supports torch.compile, so ordinary PyTorch training code has a direct path. On the serving side, the vLLM TPU backend tpu-inference unifies JAX and PyTorch models, and on 20 October 2026 native PyTorch backends for vLLM and SGLang on TPU were announced. Google's ecosystem also includes MaxText reference models and the Pathways runtime for large distributed jobs.
None of that changes one hard fact: CUDA code does not run on TPUs. If your training loop calls a hand-written fused attention kernel, a custom CUDA extension or a library that only ships CUDA binaries, that piece has to be replaced with a framework or XLA equivalent. AMD sits in between: HIP can translate much CUDA source, but performance tuning still takes time.
| What your code looks like | NVIDIA GPU | AMD Instinct GPU | Google TPU |
|---|---|---|---|
| Pure JAX model | Runs | Runs (ROCm) | Native target |
| Standard PyTorch, compiles with torch.compile | Runs | Usually runs on ROCm | Ports via TorchTPU |
| Serving with vLLM or SGLang | Runs | Runs (ROCm builds) | Supported via tpu-inference and new native backends |
| Custom CUDA kernels or extensions | Runs | Needs HIP port and tuning | Must be rewritten |
| CUDA-only third-party libraries | Runs | Depends on vendor support | Not available; find a replacement |
05 · Matching Workloads to Accelerators
Answer: Decide per workload, in order: timing, CUDA dependence, framework fit, then workload type. GPUs remain the default for anything needed before 2027 or tied to CUDA. Large dense and MoE training are TPU 8t candidates, and high-volume reasoning, agentic and long-context inference are TPU 8i candidates.
Oracle's announcement names the workloads it expects on TPUs: large-model training and post-training, both dense and mixture-of-experts, and production inference for reasoning, agentic and long-context applications. Those are also the workloads where cost per unit of work dominates the bill, which is where a purpose-built chip has the most room to pay off.
The flow below turns that into five questions you can ask about any single workload. Run it separately for training and for serving, because the same model often lands on different hardware for each stage. A model trained on GPUs can still be served on TPUs if its inference path runs through a framework both support.
Notice that a TPU never appears as an automatic answer. The best a workload can earn is a place on the shortlist, followed by a benchmark against the GPU capacity you already run. That keeps the decision grounded in your own model, data and cost per token rather than in headline figures.
| Workload | First choice today | 2027 candidate | Why |
|---|---|---|---|
| Frontier pre-training, dense or MoE | NVIDIA OCI Supercluster | TPU 8t pods | Scale and cost per training step dominate |
| Post-training and fine-tuning | NVIDIA or AMD GPUs | TPU 8t if code is JAX or TorchTPU ready | Depends on framework fit and job size |
| Reasoning and agentic inference at volume | NVIDIA or AMD GPUs | TPU 8i | Google claims up to 80% better performance per dollar for inference vs Ironwood |
| Long-context serving | GPUs with large memory | TPU 8i (288 GB HBM per chip) | Memory capacity limits context length and batch size |
| Research with new custom kernels | NVIDIA GPUs | Stay on GPU | CUDA dependence and fast iteration |
| Small models, embeddings, RAG pipelines | Right-sized GPU instances | Evaluate case by case | Accelerator choice rarely dominates the cost |
06 · Timing: Planning Around a 2027 Arrival
Answer: Treat TPUs on OCI as a 2027 option with unknown shapes, regions and prices. Do not delay production work to wait for them. Instead, keep new code portable now so that moving a workload later is a benchmark and a configuration change, not a rewrite.
Oracle says TPU availability starts in calendar year 2027 and will ramp "over a multi-year period to meet customer needs". That wording matters. Early capacity is likely to be limited, and Oracle has not said which regions get TPUs first, which TPU generations will be offered at launch, or whether orchestration through OKE or managed services will be part of the offer. The release describes the eighth generation as Google's current portfolio, but it does not commit OCI to a specific chip.
For most enterprises this means a two-track plan. Track one runs today's training and inference on OCI GPUs, sized for the next 12 to 18 months. Track two prepares the code and the operating model so that TPU capacity can be evaluated as soon as Oracle publishes shapes.
Oracle's own summary is the right planning principle: different AI workloads have different compute needs. Put each workload on the chip that fits it, and keep the code free to move.
07 · A Practical Evaluation Plan
You can start preparing for a mixed accelerator estate now, using only capacity that exists today. These steps keep you out of lock-in in both directions.
- Classify every AI workload. Record whether it is training, post-training or inference, its model type (dense or MoE), its scale, and its latency and cost targets. The decision flow above needs these facts.
- Scan for CUDA dependence. List custom kernels, extensions and CUDA-only libraries per repository, and mark which have framework or XLA equivalents.
- Make new code framework-first. Prefer JAX, or PyTorch that runs under
torch.compile, and serve through vLLM or SGLang. This code runs on NVIDIA and AMD today and has a TPU path tomorrow. - Keep data in OCI services. Store datasets and checkpoints in Object Storage and retrieval data in Oracle AI Database 26ai, so a compute move never forces a data move.
- Define a benchmark you trust. Fix a model, dataset, quality target and metric such as cost per million tokens or time to a training checkpoint, and run it on current GPU shapes now.
- Rerun it when TPU shapes publish. When Oracle announces TPU shapes and pricing, run the same benchmark and let the numbers decide placement for each workload.
Teams that follow this plan get a useful side effect even if they never adopt TPUs: cleaner, more portable code, a repeatable benchmark, and a clear view of what each workload actually costs to run.
08 · Key Takeaways
- Three accelerator families, one cloud. OCI runs NVIDIA and AMD GPUs today and adds Google TPUs in calendar 2027, ramping over several years.
- Software decides first. Framework-level JAX and PyTorch code can move between chips; custom CUDA kernels cannot run on TPUs.
- TPUs target scale. Large dense and MoE training (TPU 8t) and high-volume reasoning, agentic and long-context inference (TPU 8i) are the strongest candidates.
- Do not wait on unpublished details. Shapes, regions, prices and orchestration for TPUs on OCI are not yet announced; run production on GPUs now.
- Benchmark, then place. Build a repeatable benchmark on today's GPUs and rerun it when Oracle publishes TPU shapes.
09 · Frequently Asked Questions
Is a TPU better than a GPU for AI?
Neither is better for everything. GPUs run almost any AI code and have the broadest ecosystem. TPUs are purpose-built for large-scale neural network training and inference and aim for better scale and cost per unit of work on the workloads they target. The right choice depends on your code, model type, scale and timing.
When will Google TPUs be available on OCI?
Oracle says calendar year 2027, ramping over a multi-year period. Oracle has not yet published the launch date within 2027, regions, shape names or pricing.
Can I run my CUDA code on a TPU?
No. CUDA code does not run on TPUs. Framework-level code in JAX, or PyTorch through TorchTPU, can be ported, but custom CUDA kernels and CUDA-only libraries must be rewritten or replaced with framework or XLA equivalents.
Which workloads suit TPUs on OCI?
Oracle names large-model training and post-training, both dense and mixture-of-experts, and production inference for reasoning, agentic and long-context applications. Google positions TPU 8t for training and TPU 8i for inference and agentic work.
Should I wait for TPUs before starting an AI project on OCI?
Usually not. Run production work on OCI GPUs today, keep new code framework-first and portable, and benchmark TPUs when Oracle publishes shapes and pricing.
Does OCI still offer AMD GPUs?
Yes. AMD Instinct MI355X instances are generally available on OCI, and Oracle has announced plans for an AMD MI450 supercluster of 50,000 GPUs.
Learn this hands-on
Practice Exadata, OCI and multi-cloud architectures in live labs with a practicing consultant.
More in AI infrastructure
Inside Google's Eighth-Generation TPUs: TPU 8t for Training, TPU 8i for Inference
How the two eighth-generation TPU designs differ, what the pod sizes and memory mean, and which workloads each one targets.
Developer guideRunning AI Workloads on TPUs: JAX, PyTorch and vLLM Without the Lock-In
The TPU software stack in 2026: JAX, PyTorch through TorchTPU, vLLM and SGLang serving, and how portable your code really is.
Planning and strategyPreparing for Google TPUs on OCI in 2027: An Enterprise Readiness Guide
What to do before TPUs reach OCI in 2027: workload triage, data placement next to Oracle databases, skills, cost and governance.