Question
Loading large models from NFS is slow. How much faster is all-flash storage?
Short answer
On a Huawei Ascend 910B × 8 platform, moving model storage from NFS to FX100 cuts the serving load time of DeepSeek-32B from 691 s to 112 s (6.2×) and of DeepSeek-70B from 1399 s to 150 s (9.3×)R9.
Why NFS becomes the bottleneck
Loading a large model is essentially reading tens to hundreds of gigabytes of weight files sequentially into RAM and GPU memory. How fast that goes depends on the throughput the storage can provide.
Many existing clusters keep models on NFS because it is simple to deploy and every node can reach it. But its throughput is capped by the file server and the network path, and several nodes loading at once compete with each other. The more accelerators and the larger the models, the more visible this bottleneck becomes.
Measured results on Ascend 910B
How this relates to the AMD MI308X data
There is no conversion between them. The figures above come from the Ascend 910B platform with NFS as the baseline; the loading test on MI308X uses a different baseline and concludes that with eight GPUs cold-reading models at once, wall-clock time is 16% shorter overall and each model loads 30% fasterR1.
The two sets differ in both platform and baseline, so they cannot be added together and one cannot be used to estimate the other. Only the direction is certain: model loading is bound by storage throughput, and faster storage makes it faster.
What faster loading is good for
The most direct effect is on model switching. When many models share a pool of accelerators, every switch waits for a load; on the MI308X platform, modeling from measured load times shows effective compute utilization rising from 46.7% to 62.8% at 20 switches per hourR1.
Next is recovery time. Service restarts, node failover and elastic scale-out all have to finish loading the model before taking traffic. Cutting a load from ten or twenty-odd minutes to two or three makes a real difference to availability.
Related questions
Do these figures apply to other domestic accelerators?
Not directly. R9 was measured on the Ascend 910B platform onlyR9. Other platforms need their own measurements and get their own reports; figures from different platforms are not mixed.
Related test reports
- R1FX100 Comprehensive LLM Inference and Training Benchmark (8 × AMD MI308X)One report, five questions: how much the LMCache parallel-read patch helps, whether KV tiering scales across GPUs, how much faster 8 GPUs load models concurrently, how effective utilization changes under frequent model switching, and how much faster concurrent training checkpoint writes become. Test models are Qwen2.5-32B and 7B.
8 × AMD Instinct MI308X · ROCm 7.2 - R9FX100 on Huawei Ascend 910B: Inference and Training Benchmark (vs. NFS Baseline)Three comparison groups against an NFS baseline on an Ascend platform with 8 × Huawei Atlas 910B: model loading for inference (DeepSeek-32B / 70B), training weight and checkpoint I/O (Qwen-7B), and training-data acceleration (YOLOv8 / COCO). The platform is Ascend 910B, not MI308X, and is labeled as such wherever this site quotes it.
8 × Huawei Atlas 910B (Kunpeng-920)
External references
- 01NVM Express Specifications · NVM Express, Inc.
The official specifications of NVMe and NVMe over Fabrics
- 02RFC 5040: A Remote Direct Memory Access Protocol Specification · IETF
The normative definition of the RDMA protocol
Further reading
Low GPU Utilization From Frequent Model Switching
Each model switch leaves GPUs waiting for weights to load. At 20 switches per hour, shorter load times raise effective compute utilization from 46.7% to 62.8%.
SourceR1
Local NVMe or an NVMe-oF Array for KV Cache?
Both avoid full recompute. Across three concurrency levels on a 480B model, an external FX100 array gives more throughput and lower TTFT than one local NVMe drive.
SourceR1R2R3
How to Verify a Vendor's LLM Inference Benchmarks
Five checks for vendor inference benchmarks: baseline, platform and topology, run-to-run deviation, third-party report, reproduction material. Plus common traps.
SourceR2R3R4R5R8R9