Glossary

What Is HBM (high-bandwidth memory)?

HBM (high-bandwidth memory) is the stacked, high-bandwidth memory used on AI accelerators, in other words GPU memory. It is extremely fast but limited in capacity and expensive, and model weights and KV cache compete for it.

Updated November 12, 2026

In the measurements

Each MI308X card in the test platform on this site has 192 GB of HBM. Report R5 argues that an external KV tier can stand in for 128 GB-class HBM residency (MetaX N260 platform)R5.

Related test reports

R5FX100 KV Cache Benchmark (14B, HBM Efficiency)This report answers whether an external KV tier can stand in for GPU memory. A seven-way comparison on a 14B model (GDS direct read, recompute at two HBM sizes, direct read with low HBM) builds a three-step argument for HBM equivalence. The platform is a single MetaX N260 GPU; the method carries across platforms, but the numbers are never mixed with MI308X results.2026-07-03
Single MetaX N260 GPU

Nearby terms

RoCEv2
RoCEv2 (RDMA over Converged Ethernet v2) is a protocol that implements RDMA on Ethernet. Its packets are encapsulated in UDP/IP, so they can be routed across layer-3 networks. RDMA moves data directly between the memory of two machines, bypassing the operating-system kernel, which keeps latency and CPU usage low.
TP (tensor parallelism)
Tensor parallelism (TP) splits every layer of a model across several GPUs that compute it together. TP=8 means one model instance spans 8 GPUs, a common way to deploy models too large for a single card; TP4×2 divides 8 GPUs into two instances of 4 GPUs each.

All terms

Test jointly first, decide afterwards

Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.

Book a joint test See the evidence

Contact: Wang Limin
Phone: +86 131 6181 8898
Email: mingxin@agentmail.to
The inbox is monitored and triaged by AI around the clock