Inside Google's Eighth-Generation TPUs: TPU 8t for Training, TPU 8i for Inference

On this page

1. 01 · Two Chips Instead of One
2. 02 · Why Google Split the Generation
3. 03 · TPU 8t: Built for Training at Superpod Scale
4. 04 · TPU 8i: Built for Inference and Agents
5. 05 · TPU 8t vs TPU 8i Side by Side
6. 06 · Where Memory and Interconnect Matter: Three Worked Examples
7. 07 · What This Means for OCI Customers
8. 08 · Key Takeaways
9. 09 · Frequently Asked Questions

01 · Two Chips Instead of One

Architecture brief · Google TPU eighth generation

Why does one TPU generation now have two names?

Until 2026, each TPU generation was a single chip that Google configured for both training and serving. At Google Cloud Next on 22 April 2026, that changed. Google introduced the eighth generation as two separate designs: TPU 8t, built to train and post-train the largest models, and TPU 8i, built for inference and for what Google calls the iterative work of many specialized agents.

This matters to OCI customers because Oracle's 27 October 2026 announcement brings Google TPUs to OCI from calendar 2027, and the release describes the eighth generation as Google's current portfolio. Oracle has not said which generations will be offered on OCI at launch. Even so, the 8t and 8i split is the clearest guide available to how Google expects its chips to be used, and therefore to which of your workloads are worth evaluating.

02 · Why Google Split the Generation

Answer: Training and inference stress hardware in different ways. Training needs enormous aggregate compute and memory spread across thousands of tightly linked chips for weeks at a time. Inference needs fast access to model weights and cached context for each request, at the lowest cost per token. One chip cannot be optimal for both.

A training run is one giant, synchronized job. Every step, thousands of chips compute on their slice of the model and data, then exchange gradients before anyone can move on. The slowest link sets the pace, so training hardware lives or dies on aggregate memory and chip-to-chip bandwidth across the whole pod.

Inference is the opposite shape: millions of small, independent requests, each of which must read the model weights and its own growing context, the key-value (KV) cache, for every token it generates. Here the bottleneck is how quickly each chip can feed data to its compute units, and how many requests it can keep in memory at once. Agentic applications add a twist: many agents call the model in loops, re-reading shared context over and over, which rewards keeping hot data close to the chip.

By building two chips, Google can tune each one for its job instead of compromising. That is the same logic Oracle applies across its portfolio, just one level down: match the workload to the hardware designed for it.

03 · TPU 8t: Built for Training at Superpod Scale

Answer: TPU 8t is Google's training chip. Google says a TPU 8t superpod scales to 9,600 chips with about 2 PB of shared high-bandwidth memory and roughly 121 FP4 exaflops, nearly three times the compute per pod of the previous generation, Ironwood (TPU v7).

The superpod is the real product. Each chip has, according to Google, 19.2 Tbps of scale-up bandwidth to its neighbours inside the pod and 400 Gbps of scale-out bandwidth to the wider network, carried on a new fabric Google calls Virgo. Scale-up links keep the chips of one pod working as a single machine. The scale-out fabric connects pods to each other and to storage, which matters for loading data and writing checkpoints.

The figure below shows how a TPU 8t superpod could sit inside an OCI datacenter. Racks of TPU 8t chips are joined by the scale-up interconnect and share a pooled view of HBM. A scale-out fabric links the pod to OCI data services such as Object Storage and File Storage. OCI supplies the power, cooling and datacenter networking, as the announcement states, and your tenancy launches and monitors the job.

Two layers matter most for your job's speed: the blue scale-up interconnect, which carries gradient exchanges every step, and the shared HBM pool, which decides how large a model and batch fit. The scale-out path only needs to keep data and checkpoints flowing. How OCI will carve superpods into customer shapes is not yet published.

Oracle's own summary of the training benefit is that TPU 8t "cuts frontier model development cycles from months to weeks". Read that as a vendor claim for frontier-scale runs, and verify it against your own model when capacity arrives.

04 · TPU 8i: Built for Inference and Agents

Answer: TPU 8i is Google's inference and agentic chip. Google says each chip has 288 GB of HBM and 384 MB of on-chip SRAM, about three times the previous generation, in pods of 1,152 chips delivering about 11.6 exaflops, with up to 80% better performance per dollar for inference and up to 2x performance per watt versus Ironwood.

The headline change is memory close to the compute. On-chip SRAM is far faster than HBM but much smaller. Tripling it to 384 MB lets more of the hottest data, such as attention state and frequently reused parts of the KV cache, stay on the chip instead of making a round trip to HBM for every token. For workloads that generate long answers or run many agent turns, those round trips are a major share of the cost.

The 288 GB of HBM per chip sets how much model and context each chip can hold. By simple multiplication, a full 1,152-chip pod holds about 330 TB of HBM in total. Larger pods, up from 256 chips in the previous generation, give Google room to split very large or mixture-of-experts models across many chips while keeping the traffic inside one pod.

Google positions TPU 8i for reasoning models, agentic systems and long-context applications, the same inference workloads Oracle names in its announcement. These are the workloads where cost per token, not peak compute, decides the business case.

05 · TPU 8t vs TPU 8i Side by Side

Answer: Choose TPU 8t when the job is one large training or post-training run that needs maximum aggregate memory and chip-to-chip bandwidth. Choose TPU 8i when the job is serving many requests or agents, where memory close to the compute and cost per token matter most.

The two chips share a generation and a software stack, JAX, PyTorch through TorchTPU and the XLA compiler, so the same model code can train on one and serve on the other. What differs is the balance of memory, pod size and compute each is built around. The comparison below uses only Google's published figures for each design.

The pattern is easy to remember. TPU 8t numbers are pod-scale totals, because training treats the pod as one machine. TPU 8i numbers are per chip and per pod, because inference cares about how much each chip can hold and how cheaply it can answer. Neither figure set tells you OCI shapes or prices.

06 · Where Memory and Interconnect Matter: Three Worked Examples

Answer: In training, memory sets how big a model and batch fit, and the scale-up interconnect sets how fast each step finishes. In inference, HBM capacity sets how many requests and how much context fit per chip, and SRAM plus HBM bandwidth set how fast each token is produced.

The figure below walks through three paths: one training step across a TPU 8t pod, one inference request on TPU 8i, and a multi-agent loop on TPU 8i. Yellow tags mark the points where memory is the constraint and blue tags mark where interconnect is. The arithmetic that follows uses common rules of thumb, not measured results, and is meant only to show the scale involved.

The loop in the third lane is the key difference between chat and agents. A chat request runs once. An agent workflow returns to the model again and again with largely the same context, so memory that keeps that context close to the compute pays off on every turn.

1. Training step on TPU 8t. A common rule of thumb for mixed-precision training with the Adam optimizer is about 16 bytes per parameter for weights, gradients and optimizer state. A 1-trillion-parameter model therefore needs roughly 16 TB before activations, under 1% of the 2 PB Google cites per superpod. The rest of the memory goes to activations and larger batches, while the 19.2 Tbps per-chip scale-up links carry the gradient exchange that ends every step.
2. Single inference request on TPU 8i. A 70-billion-parameter model stored at 8-bit precision needs about 70 GB for weights, so it fits on one 288 GB chip with roughly 200 GB left for KV caches. More free HBM means more concurrent requests or longer contexts per chip. During decode, the 384 MB of SRAM keeps the hottest data on the chip, which is where per-token speed is won.
3. Multi-agent loop on TPU 8i. A planner agent splits a task, specialist agents call tools and retrieval, and every agent returns to the model with shared context. If ten agents each run five turns against the same context, that context is read fifty times. Keeping it resident in HBM and SRAM, rather than rebuilding it, is what makes the loop affordable at scale.

07 · What This Means for OCI Customers

Answer: Use the 8t and 8i split to sort your AI workloads into training and serving buckets now. When Oracle publishes TPU shapes for OCI, you will know which shape family to benchmark for each workload, and which figures to ask Oracle about.

Oracle's announcement names exactly the workloads these two chips target: large-model training and post-training, dense and mixture-of-experts, plus production inference for reasoning, agentic and long-context applications. OCI provides the power, cooling and datacenter networking, and availability starts in calendar 2027, ramping over several years.

What is missing is everything that turns a chip into a cloud service: the shapes you can rent, the pod slice sizes, the regions, the pricing, and how jobs will be scheduled. Keep your planning documents honest by recording those items as open questions.

Google's Amin Vahdat said the next generation of AI "will require a diverse range of compute." Splitting one TPU generation into a training chip and an inference chip is that idea applied inside a single product line.

08 · Key Takeaways

1. Two chips, one generation. Google's eighth generation splits into TPU 8t for training and TPU 8i for inference and agents.
2. TPU 8t is about the pod. Google cites up to 9,600 chips, about 2 PB of shared HBM and roughly 121 FP4 exaflops per superpod, with 19.2 Tbps scale-up links per chip.
3. TPU 8i is about memory per chip. Google cites 288 GB HBM and 384 MB SRAM per chip in 1,152-chip pods, aimed at cost per token for reasoning and agents.
4. Memory and interconnect decide speed. Training is bound by pod memory and gradient exchange; inference by HBM capacity and how close hot data sits to compute.
5. OCI details are still open. Generations at launch, shapes, regions and pricing for TPUs on OCI are not yet published; plan with Google's figures as a guide only.

09 · Frequently Asked Questions