LPUs & Wafer-Scale Engines

Available

Two approaches to breaking the memory wall for AI: Groq's deterministic LPU for low-latency inference, and Cerebras' wafer-scale engine for massive on-chip SRAM.

Overview

The Language Processing Unit (LPU) takes a deterministic, statically-scheduled approach to inference: no caches, no speculative execution, a compiled dataflow that yields highly predictable Time-To-First-Token. This makes it well suited to latency-sensitive, interactive inference.

The Wafer-Scale Engine (WSE) takes the opposite extreme on capacity: an entire wafer as one chip, with enormous on-chip SRAM that keeps model weights resident and removes the DRAM round-trips that bottleneck large-model training.

Key Pain Points

DRAM bottleneck

Moving weights between DRAM and compute dominates cost and latency in large models.

Latency variance

Cache and speculation make GPU inference latency hard to predict for interactive use.

Training scale

Distributing very large models across many GPUs adds communication and orchestration overhead.

Energy per token

Data-movement energy dominates; keeping state on-chip reduces it.

Methods & Fit

Where this architecture addresses the pain points above:

Deterministic scheduling (LPU)

A compiled, statically-scheduled dataflow yields predictable low-latency inference.

On-chip SRAM (WSE)

Wafer-scale SRAM keeps weights resident, removing DRAM round-trips for training.

Model-parallel simplicity

A single large device can reduce the multi-GPU distribution and communication overhead.

Typical workload classes:

Low-latency chat Real-time agents Large-model training High-throughput inference Scientific AI Batch generation

Roadmap

Phase	Strategic Action	Outcome
1. Latency Audit	Profile Time-To-First-Token on current GPU clusters vs. LPU specifications.	Benchmark baseline for ROI.
2. Memory Mapping	Optimise model weights for wafer-scale SRAM residency.	Elimination of DRAM bottlenecks.
3. Pilot Deployment	Run a latency-sensitive inference or a large-model training pilot.	Validated performance case.
4. Production Scale	Integrate into production inference or training environments.	Predictable AI performance.

Metrics

Sub-second

deterministic Time-To-First-Token (LPU)

Wafer-scale

one wafer as a single chip with massive SRAM (WSE)

Available

both platforms are in production use today

Limitations

Specialised targets: the LPU is inference-oriented and the WSE is training/HPC-oriented; neither is a general-purpose replacement.
Toolchain specificity: both require vendor compilers and model mapping; portability differs from commodity GPUs.
Fit-dependent economics: the advantage depends on matching the workload profile to the right platform.

This page assesses technical fit, not a procurement decision.

Overview Request an architecture audit