Exadata Exascale Architecture Explained: Elastic Compute and Storage for Oracle Databases

On this page

1. 01 · Why Exadata Needed a New Architecture
2. 02 · The Big Picture: Pools, Not Racks
3. 03 · The Layered Stack, Top to Bottom
4. 04 · Storage Vaults and Exascale Files
5. 05 · How I/O Flows Through Exascale
6. 06 · Elastic Compute and Storage in Practice
7. 07 · Snapshots and Thin Clones for DBAs
8. 08 · What Stays the Same, and What Changes
9. 09 · Key Takeaways
10. 10 · Frequently Asked Questions

01 · Why Exadata Needed a New Architecture

Architecture deep dive · Exadata Exascale

"We want Exadata, but we only need a slice of one."

For most of its history the unit of Exadata was the rack, or at least a set of whole database and storage servers. That suited consolidated ERP and data warehouse estates, and it made life hard for everyone else. A team with one 2 TB database, or a fleet of short-lived test copies, had to choose between overbuying Exadata and giving up Smart Scan, RDMA and the storage software that make it special.

Exascale is Oracle's answer. It is not a smaller Exadata; it is a different way of running the same Exadata hardware and software. Oracle operates large shared pools of database servers and storage servers, and each customer gets a VM cluster and a storage vault carved out of those pools. In the cloud it is sold as Exadata Database Service on Exascale Infrastructure (ExaDB-XS) on OCI and through Oracle AI Database@AWS, @Azure and @Google Cloud.

02 · The Big Picture: Pools, Not Racks

Answer: Exascale runs many customers' VM clusters on a shared pool of Exadata database servers, and stores their databases in storage vaults spread across a shared pool of Exadata storage servers. An RDMA network fabric connects the two pools, and Exascale control services manage vaults, files, clones and pool health.

Three ideas define the architecture. First, the database servers are a pool: your VM cluster's virtual machines run on servers that also host other tenants' VMs, with hypervisor isolation and per-VM ECPU allocation. Second, the storage servers are a pool too: every vault's data is spread across all of them, so even a small database gets the I/O bandwidth of many servers. Third, a set of Exascale services replaces the per-cluster ASM instance and keeps the metadata about which file lives where.

The topology below shows one Exascale site with three tenants. Each tenant connects from its own network to its own VM cluster. Below the RDMA fabric, the coloured stripes show the three tenants' vaults, each spread across all four storage servers.

Follow tenant A: its VM cluster runs on database servers 1 and 2, and its vault stripe crosses every storage server. That is the core of the elasticity story. Adding ECPUs grows the VMs on the top pool; adding gigabytes grows the stripe on the bottom pool. Neither change requires new hardware for that tenant.

03 · The Layered Stack, Top to Bottom

Answer: Exascale has seven practical layers: the Oracle Database instance, the VM cluster, the Exascale data access layer in the database, the RDMA fabric, the Exascale storage services with vaults and files, the Exadata storage server software, and the physical memory, flash and disk tiers.

Most of these layers exist on any Exadata. What Exascale changes is the middle: the database no longer talks to an ASM instance to learn where extents are. It asks the Exascale services for file metadata and then reads and writes storage servers directly. VM images are also stored on Exascale volumes rather than on local disks, which is what lets Oracle place and move VMs within the database server pool.

The bracket on the right is the operating model in one picture. You manage the database layer: schemas, users, PDBs, tuning and patch timing. Oracle manages everything from the VM down. The two yellow tags mark the only two knobs that change your bill: ECPUs on the VM layer, gigabytes on the vault layer.

04 · Storage Vaults and Exascale Files

Answer: A storage vault is a logical container for database files in the Exascale storage pool. It has a capacity you pay for in gigabytes, its files are striped and mirrored across all storage servers, and databases reference it directly by name instead of through an ASM disk group.

If you think in ASM terms, a vault plays the role of the disk group from the database's point of view, but without disks you own. When you create an ExaDB-XS VM cluster you associate it with a vault, and database files created there carry the vault name in their path, prefixed with an at sign. Commands such as CREATE TABLESPACE and parameters such as DB_CREATE_FILE_DEST simply point at the vault.

What happens inside the pool

Each Exascale file is split into extents that are distributed across the storage servers in the pool and kept with redundant copies on different servers. Oracle handles rebalancing when servers are added or replaced, the job that ASM rebalance used to do on dedicated systems. Because extents of every vault land on every server, an I/O-heavy query on a small database still fans out across the whole pool.

05 · How I/O Flows Through Exascale

Answer: Exascale keeps Exadata's three I/O superpowers: Smart Scan pushes filtering to the storage servers, RDMA lets the database read hot blocks straight from storage server memory, and snapshots let new databases share existing extents. The walkthrough shows each path step by step.

Performance questions about Exascale usually come down to one worry: does a shared pool mean slower I/O? The answer depends on the I/O type, so it helps to trace the three most common ones. The figure follows a reporting query, a single-row OLTP lookup and a thin clone from request to result.

The green tags at the bottom are the takeaway. Smart Scan returns only matching rows, so the RDMA fabric carries far less data than the table size. The OLTP read never wakes the storage server's CPU when the block is cached in memory. The clone writes metadata, not data, so its time does not grow with database size.

Smart Scan on a shared pool

A full scan with a selective predicate is sent to every storage server holding extents of the table. Storage indexes skip regions whose min and max values cannot match, the remaining blocks are filtered and projected on the storage servers, and only qualifying rows and columns return. Because the vault is spread across the whole pool, the scan gets parallelism from many servers even for a modest table.

OLTP reads over RDMA

For a single-block read the database checks the buffer cache first, then reads the block directly from Exadata RDMA Memory or flash cache on the storage server that holds it, using RDMA over the RoCE fabric. If the block is not cached, the storage server reads it from flash or disk. This is why Exascale OLTP latency stays in the Exadata class rather than the network storage class.

Thin clones

A clone request creates new file entries that point at the same extents as the source. Only blocks changed afterwards consume new space. A 5 TB PDB can therefore be cloned for testing in seconds and cost little until the test workload starts writing.

06 · Elastic Compute and Storage in Practice

Answer: Compute and storage scale independently. You change the ECPU count of a VM cluster or the capacity of its storage vault through the console, CLI or Terraform, and pay for what is allocated, with no separate charge for IOPS.

Independent scaling changes capacity planning. On dedicated infrastructure, you size the rack for the peak of both CPU and storage, and live with whichever is idle. On Exascale the two curves are separate. A reporting database that needs 40 TB but little CPU, and a busy OLTP schema that needs many ECPUs but 500 GB, are both sized honestly.

1. Start small. Provision the VM cluster with the ECPUs the current workload needs and a vault sized to today's data plus growth headroom.
2. Watch the right metrics. Track CPU utilization, average active sessions and vault used capacity in OCI monitoring or your hyperscaler's monitoring service.
3. Scale ECPUs for peaks. Increase ECPUs before month end or a launch, then reduce them afterwards. Database services stay up during the change.
4. Grow the vault ahead of need. Increase vault capacity before free space runs low; there are no cells or disks to order.
5. Review monthly. Compare allocated ECPUs and gigabytes with actual use and adjust. Elasticity saves money only if someone turns the dial down.

07 · Snapshots and Thin Clones for DBAs

Answer: Exascale makes snapshot and clone operations native to storage. A PDB snapshot copy or a database clone shares extents with its source and consumes space only for changed blocks, which makes per-developer and per-test-run copies affordable.

On dedicated Exadata, thin cloning needed sparse disk groups, a read-only test master and a careful refresh routine. On Exascale the storage layer does the work. You can still use familiar SQL, and the cloud consoles expose clone operations for whole databases.

-- Thin clone of a production PDB for testing (same CDB)
CREATE PLUGGABLE DATABASE sales_test FROM sales_prod SNAPSHOT COPY;
ALTER PLUGGABLE DATABASE sales_test OPEN;

-- Where are the files? Exascale paths start with the vault name
SELECT name FROM v$datafile WHERE con_id = (SELECT con_id FROM v$pdbs WHERE name = 'SALES_TEST');

Good practice is to treat clones as disposable. Build them from a masked copy if they leave the production security boundary, tag them with an owner and an expiry date, and drop them when the test cycle ends. Clones are cheap to create, and their changed blocks are still billed vault capacity.

08 · What Stays the Same, and What Changes

For the people who write SQL and run batch jobs, very little changes. The database release, the optimizer, RAC, Data Guard, AWR and the Exadata performance features behave as they do on dedicated systems. For the people who plan capacity and run operations, quite a lot changes.

• No ASM administration. Disk groups, rebalance power and failure groups are no longer your concern.
• Capacity is a setting. ECPUs and vault size replace hardware orders and rack elevations.
• Release requirements. Exascale supports Oracle AI Database 26ai and Oracle Database 19c (19c through Exascale volumes with ASM), so check the release updates on offer and your application certification first.
• Noisy neighbours are Oracle's problem. Resource isolation in the shared pools is part of the service.
• Clones become routine. Test, development and analytics copies can be created per sprint instead of per quarter.

Exascale does not make Exadata smaller. It makes the unit you rent smaller, while every query still runs on the full power of the pool underneath.

09 · Key Takeaways

1. Two shared pools, one fabric. Exascale runs tenants' VM clusters on a shared database server pool and their vaults on a shared storage server pool, joined by RDMA.
2. Vaults replace ASM. Database files live in storage vaults, are striped across every storage server and are referenced by vault name.
3. Exadata features are intact. Smart Scan, storage indexes, flash caching and RDMA reads still do the heavy lifting.
4. Scale each dimension on its own. ECPUs and storage gigabytes are separate dials, with no IOPS charge.
5. Clones are metadata. Snapshot copies and thin clones share extents and cost space only for changed blocks.

10 · Frequently Asked Questions