Reconfigurable Dataflow (RDU)
AvailableSambaNova's Reconfigurable Dataflow Unit maps entire model graphs onto the hardware, targeting high-throughput enterprise inference and RAG.
Overview
Reconfigurable Dataflow architectures map the operator graph of a model directly onto a reconfigurable fabric, rather than streaming instructions through fixed cores. For large-model inference this reduces the overhead of moving weights and activations repeatedly through a memory hierarchy.
The practical target is enterprise inference — retrieval-augmented generation, batch inference and agentic pipelines — where sustained throughput and predictable cost per token matter more than single-request latency.
Key Pain Points
Memory-hierarchy overhead
Repeatedly moving weights through DRAM/cache limits large-model throughput.
Cost per token
Enterprise inference economics depend on predictable, low cost per token at scale.
Large context windows
RAG and long-context workloads stress memory capacity and bandwidth simultaneously.
Deployment complexity
Standing up multi-model enterprise inference on GPU clusters is operationally heavy.
Methods & Fit
Where this architecture addresses the pain points above:
Graph-to-fabric mapping
Maps the model's operator graph onto the reconfigurable fabric to cut instruction and data-movement overhead.
Large on-chip state
Keeps more model state resident to reduce trips through external memory.
Multi-model serving
Targets serving several models on one platform for enterprise and RAG pipelines.
Typical workload classes:
Roadmap
| Phase | Strategic Action | Outcome |
|---|---|---|
| 1. Workload Audit | Profile inference throughput and cost-per-token targets vs. current GPU baseline. | ROI baseline. |
| 2. Model Mapping | Map target models onto the RDU fabric and validate accuracy parity. | Validated model set. |
| 3. Pilot Serving | Deploy a RAG or batch-inference pilot and measure sustained throughput. | Throughput report. |
| 4. Production Scale | Scale multi-model serving into production with monitoring. | Scalable enterprise inference. |
Metrics
optimised for sustained tokens/s over single-request latency
several models served on one platform
targeted at retrieval-augmented and long-context workloads
Limitations
- Inference-focused: the platform targets inference and serving, not primarily large-scale training.
- Ecosystem lock-in: mapping and tooling are vendor-specific; portability differs from commodity GPU stacks.
- Workload fit: benefits are clearest for high-throughput serving; low-volume or highly interactive workloads may not justify it.
This page assesses technical fit, not a procurement decision.