// Knowledge Base

High-Performance Computing
Lexicon A–Z

The definitive HPC reference for supercomputing professionals, researchers, and architects. Covering everything from MPI and SLURM to exascale systems, InfiniBand fabrics, and GPU clusters.

350+
HPC Terms
26
Categories A–Z
2026
Continuously Updated
ABCD EFGH IJKL MNOP QRST UVWX
// In-depth articles
Deep Dive
MPI — Message Passing Interface
The standard for parallel programming across distributed memory systems. Point-to-point, collective operations, and MPI-4 innovations explained.
Deep Dive
SLURM Workload Manager
How the world's leading HPC job scheduler works. Partitions, QOS, GPU allocation, and best practices for cluster operators.
Deep Dive
InfiniBand Interconnects
HDR, NDR, and XDR generations. RDMA, latency, bandwidth, and why InfiniBand dominates the TOP500.
Deep Dive
Lustre Parallel Filesystem
The leading HPC storage system. MDT, OST, striping, and tuning for maximum I/O throughput on petabyte-scale datasets.
Deep Dive
GPU Computing in HPC
NVIDIA H100, AMD MI300X, CUDA vs ROCm, NVLink, and how GPU clusters power modern AI training and simulation.
Deep Dive
Exascale Computing
Frontier, Aurora, and the challenges of building 10¹⁸ FLOP/s machines. Power, cooling, memory, and programming models.
A
Accelerator
A specialized processor (GPU, FPGA, TPU) designed to offload and accelerate specific computational workloads from the CPU.
All-to-All Communication
An MPI collective operation where every process sends data to and receives data from every other process simultaneously.
Amdahl's Law
A formula predicting the maximum speedup of a parallel program: S = 1 / (1 - p + p/n), where p is the parallelizable fraction and n is the number of processors.
Application Checkpointing
Periodically saving the state of a running application so computation can resume from that point after a failure — critical for long HPC jobs.
ARM Architecture in HPC
RISC-based processor architecture increasingly used in HPC — Fujitsu A64FX powers Fugaku, the world's most energy-efficient top supercomputer.
Allreduce
MPI collective combining reduction (sum, max, etc.) across all processes with the result broadcast back to all — the most critical operation in distributed AI training.
B
Bandwidth
The maximum rate of data transfer across a network or memory bus, measured in GB/s or TB/s. A key bottleneck in HPC systems.
Batch Job
A non-interactive job submitted to a workload manager (SLURM, PBS) that runs when resources are available, without real-time user interaction.
Bisection Bandwidth
The minimum bandwidth across any partition that divides a network into two equal halves — a measure of network topology quality.
Brownout / Brownfield HPC
Upgrading or expanding existing HPC infrastructure rather than building a new system from scratch.
BFloat16 (BF16)
A 16-bit floating point format with the same exponent range as FP32 but reduced mantissa — preferred for AI training on modern HPC accelerators.
C
Cluster
A group of interconnected compute nodes working together as a single system, sharing resources and managed by a workload scheduler.
CUDA (Compute Unified Device Architecture)
NVIDIA's parallel computing platform and programming model enabling general-purpose computing on NVIDIA GPUs.
Cache Coherency
Ensuring all processor caches see a consistent view of shared memory — a major challenge in multi-socket and multi-node HPC systems.
Cold Storage
Archival storage tier for infrequently accessed data — typically tape libraries in HPC centers, offering lowest cost per TB.
Compute Node
A server in an HPC cluster dedicated to running application workloads, typically containing CPUs, memory, and optionally GPUs.
Collective Operations
MPI operations involving all processes in a communicator: Broadcast, Scatter, Gather, Reduce, Allreduce, Barrier, and Alltoall.
Containerization (HPC)
Using Singularity/Apptainer or Docker-compatible containers to package HPC applications with their dependencies for reproducibility.
CPU Affinity
Binding processes or threads to specific CPU cores to minimize cache misses and NUMA overhead in multi-core systems.
D
Data Parallelism
Distributing different subsets of data across multiple processors, each running the same computation — the most common AI training parallelism strategy.
Dragonfly Topology
A network topology used in large HPC systems (Cray/HPE Slingshot) with groups of switches connected by all-to-all links — optimized for global bandwidth.
DGEMM
Double-precision General Matrix Multiply — the core BLAS operation used to benchmark HPC systems and the basis of the LINPACK TOP500 test.
Disaggregated Memory
Pooling memory resources across nodes in a fabric (CXL) so workloads can access more memory than physically attached to a single node.
Dynamic Voltage and Frequency Scaling (DVFS)
Adjusting CPU/GPU voltage and frequency at runtime to balance performance and power consumption.
E
Exascale Computing
Systems capable of 10¹⁸ (one quintillion) floating-point operations per second. Frontier (ORNL) was the first to cross this threshold in 2022. → Deep dive
Ethernet (HPC)
High-speed Ethernet (25/100/400GbE) used as a cost-effective interconnect alternative to InfiniBand for moderate-scale HPC and AI clusters.
Environment Modules
A system for dynamically modifying the user environment to load/unload software packages, compilers, and MPI libraries (Lmod, TCL modules).
ECC Memory
Error-Correcting Code memory that detects and corrects single-bit errors — mandatory in HPC to ensure computation reliability over long runtimes.
F
FLOPS (Floating Point Operations Per Second)
The standard measure of HPC system performance. Modern systems range from TFLOPS (10¹²) to EFLOPS (10¹⁸).
Fat-Tree Topology
A network topology where switch bandwidth increases toward the root, providing full bisection bandwidth — standard in InfiniBand HPC clusters.
Filesystem (Parallel)
Distributed storage systems (Lustre, GPFS/Spectrum Scale, DAOS) designed for concurrent high-throughput access from thousands of nodes.
FPGA (Field-Programmable Gate Array)
Reconfigurable hardware that can be programmed for specific workloads — used in HPC for financial computing, genomics, and network offload.
Frontier
The world's first certified exascale system (1.1 EFLOPS), located at Oak Ridge National Laboratory. Uses AMD EPYC CPUs and MI250X GPUs.
G
GPU (Graphics Processing Unit)
Massively parallel processor with thousands of cores optimized for matrix operations — the dominant accelerator for both HPC simulation and AI training.
GPFS (IBM Spectrum Scale)
IBM's high-performance parallel filesystem used in many TOP500 systems, offering POSIX compliance and global namespace across sites.
Gustafson's Law
An alternative to Amdahl's Law stating that parallel speedup scales with problem size — more optimistic for real-world HPC workloads.
GigaFLOPS / TeraFLOPS / PetaFLOPS
10⁹ / 10¹² / 10¹⁵ floating point operations per second. A modern NVIDIA H100 GPU delivers ~2,000 TFLOPS (FP16 Tensor Core).
H
HPC (High-Performance Computing)
The aggregation of computing power delivering performance orders of magnitude greater than typical desktop computers, used to solve complex scientific and engineering problems.
HBM (High Bandwidth Memory)
3D-stacked memory architecture delivering multi-TB/s bandwidth, used in modern AI accelerators (H100: 3.35 TB/s HBM3) and HPC processors.
Hybrid MPI/OpenMP
Combining MPI for inter-node communication with OpenMP for intra-node thread parallelism — the dominant programming model for modern HPC clusters.
HPL (High-Performance LINPACK)
The benchmark used to rank the TOP500 supercomputers — measures sustained performance on dense linear algebra (LU factorization).
Heat Dissipation / Liquid Cooling
Modern HPC systems use direct liquid cooling (DLC), immersion cooling, or rear-door heat exchangers to handle power densities exceeding 100 kW/rack.
I
InfiniBand
A high-speed, low-latency interconnect fabric standard dominant in HPC. HDR = 200 Gb/s, NDR = 400 Gb/s per port. Used in the majority of TOP500 systems. → Deep dive
I/O (Input/Output) Bottleneck
The performance gap between compute speed and storage speed — often the limiting factor in data-intensive HPC workloads like climate modeling or genomics.
ISA (Instruction Set Architecture)
The interface between hardware and software defining available instructions: x86-64 (Intel/AMD), ARM (Fugaku), and RISC-V (emerging).
Interconnect Fabric
The network connecting all nodes in an HPC cluster — InfiniBand, Slingshot, OmniPath, or high-speed Ethernet. Determines latency and bandwidth.
J
Job Scheduler
Software managing resource allocation and job queuing on HPC clusters. SLURM dominates; PBS/Torque and LSF are also widely used.
Job Array
A SLURM feature for submitting many similar jobs as a single submission — ideal for parameter sweeps and ensemble simulations.
Jitter (Network)
Variation in packet latency across an interconnect. Low jitter is critical for MPI collective operations — even microsecond variations can stall thousands of nodes.
K
Kernel (OS)
The core of the operating system managing hardware resources. HPC systems commonly use custom-tuned Linux kernels (RHEL, Rocky, Ubuntu) with minimal overhead.
KNL (Knights Landing)
Intel Xeon Phi manycore processor (up to 72 cores) designed for HPC — discontinued but important in HPC history for pushing core counts.
L
Latency
The time delay for a message or operation to complete. InfiniBand achieves sub-microsecond latency (~600ns) vs Ethernet (~5µs+).
LINPACK
The benchmark solving a dense system of linear equations used to rank TOP500 supercomputers since 1993.
Load Balancing
Distributing work evenly across compute resources to maximize utilization and minimize idle time.
Login Node
A dedicated server providing user access to an HPC cluster for job submission, file management, and compilation — never used for computation.
Lustre
The most widely deployed parallel filesystem in HPC, used by the majority of TOP10 supercomputers. Delivers millions of IOPS and TB/s throughput. → Deep dive
Liquid Cooling
Using water or dielectric fluid to cool HPC components — direct liquid cooling (DLC) and immersion cooling handle densities impossible with air.
M
MPI (Message Passing Interface)
The de facto standard for distributed-memory parallel programming in HPC. MPI-4 (2021) introduced persistent collectives and large count support. → Deep dive
Memory Bandwidth
The rate at which data can be read/written from memory. Often the limiting factor in HPC — H100 HBM3 delivers 3.35 TB/s vs DDR5 ~96 GB/s.
Mellanox / NVIDIA Networking
The dominant InfiniBand hardware vendor (acquired by NVIDIA in 2020), producing ConnectX adapters and Quantum switches for HPC fabrics.
Model Parallelism
Splitting neural network layers across multiple GPUs — necessary when model parameters exceed a single GPU's memory capacity.
Multi-Rail Networking
Connecting nodes to multiple network ports or switches for increased bandwidth and redundancy.
MLPerf (HPC)
The standardized AI training benchmark suite for HPC systems — measures time to train ResNet, BERT, GPT-3, and other models to target accuracy.
N
NUMA (Non-Uniform Memory Access)
Memory architecture in multi-socket systems where access time varies based on memory location relative to the CPU — critical for HPC performance tuning.
NVLink
NVIDIA's high-bandwidth GPU-to-GPU interconnect (900 GB/s in H100 NVLink 4.0) enabling fast gradient sharing in multi-GPU training nodes.
NVMe (Non-Volatile Memory Express)
PCIe-connected flash storage protocol with microsecond latency and multi-GB/s throughput — used for burst buffers and node-local storage in HPC.
Node
A single server in an HPC cluster — typically containing 2–4 CPUs, 256GB–2TB RAM, optional GPUs, and a high-speed network adapter.
O
OpenMP
An API for shared-memory parallel programming using compiler directives — used for intra-node thread parallelism alongside MPI.
OpenACC
A directive-based parallel programming model for offloading computation to GPUs and other accelerators.
OST (Object Storage Target)
A Lustre storage server providing object-based data storage. Large clusters may have hundreds of OSTs for aggregate bandwidth.
Overhead (Parallel)
Time spent on coordination, synchronization, and communication in parallel programs that reduces effective speedup.
P
Parallel Computing
Simultaneously executing multiple computations by dividing a problem across many processors — the foundational concept of HPC.
Pipeline Parallelism
Distributing neural network layers across GPU groups, with micro-batches flowing through the pipeline — reduces memory per GPU for large models.
Power Usage Effectiveness (PUE)
The ratio of total data center power to IT equipment power. World-class HPC facilities achieve PUE of 1.03–1.1. Lower is better.
Profiling (HPC)
Measuring where time is spent in an application using tools like Intel VTune, NVIDIA Nsight, Perf, or TAU to identify bottlenecks.
Petascale
Computing at 10¹⁵ FLOPS. Systems like Titan and Jaguar ushered in the petascale era (2008–2022) before exascale systems arrived.
Q
QoS (Quality of Service)
SLURM policies controlling priority, resource limits, and scheduling constraints for different user groups and project allocations.
Queue (Job Queue)
A collection of submitted jobs waiting for resource allocation. SLURM organizes queues into partitions with different node types and time limits.
R
RDMA (Remote Direct Memory Access)
Directly accessing another node's memory without involving the remote CPU or OS — fundamental to InfiniBand's ultra-low latency.
ROCm
AMD's open-source GPU computing platform, alternative to NVIDIA CUDA — used on Frontier's MI250X GPUs and Mi300X clusters.
Rocky Linux
The leading RHEL-compatible Linux distribution for HPC clusters following CentOS's discontinuation — binary compatible with enterprise RHEL.
Run Queue
The list of processes waiting to execute on a CPU — low run queue depth is desirable for interactive and latency-sensitive HPC workloads.
S
SLURM (Simple Linux Utility for Resource Management)
The world's most widely deployed HPC workload manager. Manages job scheduling, resource allocation, and accounting on over 60% of TOP500 systems. → Deep dive
Scalability
How well performance grows with additional resources. Strong scaling: fixed problem, more processors. Weak scaling: problem grows proportionally.
Scratch Storage
High-performance temporary storage on the parallel filesystem for active jobs — not backed up, purged regularly. Distinct from home/project storage.
Slingshot (HPE)
HPE's high-speed interconnect for Cray supercomputers (200 Gb/s, Ethernet-compatible). Used in Frontier and other leadership systems.
Symmetric Multiprocessing (SMP)
Multiple CPUs sharing a single memory space — the standard architecture for HPC compute nodes (2–4 sockets, shared DRAM).
Striping (Lustre)
Distributing a file's data across multiple OSTs for parallel I/O access. Wider stripe count = higher bandwidth for large files.
T
Tensor Parallelism
Splitting individual weight matrices across GPUs — used in Megatron-LM and similar frameworks for training very large transformer models.
TOP500
The biannual ranking of the world's 500 most powerful supercomputers based on HPL LINPACK benchmark results. Published since 1993.
TCO (Total Cost of Ownership)
Full lifecycle cost of HPC infrastructure including hardware, power, cooling, space, administration, and software licensing.
Topology (Network)
The physical and logical arrangement of network connections: Fat-Tree, Dragonfly, Torus, and Mesh each offer different bandwidth/latency tradeoffs.
Thread Pinning
Binding threads to specific CPU cores to prevent OS migration, ensuring cache locality and predictable NUMA behavior.
U
UCX (Unified Communication X)
A high-performance communication framework providing a common API across InfiniBand, shared memory, and Ethernet — used by OpenMPI and MPICH.
Utilization (Cluster)
The percentage of available compute resources actively running jobs. Well-managed HPC centers target 85–95% utilization.
V
Vectorization
Using SIMD (AVX-512, ARM SVE) instructions to perform the same operation on multiple data elements simultaneously — critical for HPC application performance.
Virtual Machine (HPC)
Hypervisor-based isolation used cautiously in HPC — bare-metal is preferred for performance, but VMs enable multi-tenancy in cloud HPC.
W
Warm Cache
A processor cache already populated with relevant data from a previous run — subsequent runs benefit from lower cache miss rates.
Walltime
The maximum real-time duration a job is allowed to run, as specified in the SLURM job script. Exceeding walltime causes automatic job termination.
Workload Manager
Software allocating shared HPC resources among competing jobs: SLURM, PBS Professional, LSF, and Grid Engine are the leading solutions.
X
Xeon Scalable (Intel)
Intel's server processor family for HPC — Sapphire Rapids (4th Gen) supports HBM, PCIe 5.0, and CXL for next-generation HPC memory architectures.
XPU
Intel's branding for heterogeneous compute covering CPUs, GPUs, and FPGAs — part of the oneAPI programming model for unified HPC development.