Solutions
Enablement for Domestic and Non-NVIDIA Accelerators
Accelerator enablement
Updated
The problem
Buying domestic or non-NVIDIA accelerators is only the first step. Inference frameworks must be rebuilt, cache-library read paths rewritten and multimodal operators verified one by one. Until someone does that work, the cards are just hardware in a rack.
Our approach
Key points and sources
- Source-level builds and patches of vLLM / LMCache on ROCm: the parallel-read patch improves TTFT by 4.1×R1
- Measured on several platforms: R1–R4 on MI308X, R9 on Ascend 910B, R5 on MetaX N260
- Multimodal works as well: all 7 ComfyUI + LTX-Video 2.3 models run on MI308X / ROCm 7.2 with no operator or kernel errorsR6R7
- Delivered down to the code: git patches, load client, orchestration and forensic scripts, and the raw dataR8
Related test reports
- R1FX100 Comprehensive LLM Inference and Training Benchmark (8 × AMD MI308X)One report, five questions: how much the LMCache parallel-read patch helps, whether KV tiering scales across GPUs, how much faster 8 GPUs load models concurrently, how effective utilization changes under frequent model switching, and how much faster concurrent training checkpoint writes become. Test models are Qwen2.5-32B and 7B.
8 × AMD Instinct MI308X · ROCm 7.2 - R5FX100 KV Cache Benchmark (14B, HBM Efficiency)This report answers whether an external KV tier can stand in for GPU memory. A seven-way comparison on a 14B model (GDS direct read, recompute at two HBM sizes, direct read with low HBM) builds a three-step argument for HBM equivalence. The platform is a single MetaX N260 GPU; the method carries across platforms, but the numbers are never mixed with MI308X results.
Single MetaX N260 GPU - R6ComfyUI + LTX-Video 2.3 Full-Model Deployment, Adaptation and Usage Report (V2)Proof of adaptation for video generation: all 7 delivered models (3 VAEs, the fp8 text encoder, the text projection and 2 LoRAs) run end to end on MI308X / ROCm 7.2 with no dtype, operator or kernel errors.
AMD Instinct MI308X · ROCm 7.2 - R7ComfyUI + LTX-Video 2.3 Model Adaptation Report (AMD MI308X)The per-model detail behind R6: individual run records for the 3 VAEs and the Singularity LoRA, plus the adaptation points that matter on MI308X.
AMD Instinct MI308X - R8FX100 KV Cache Test Code Export and Raw Benchmark DataEverything a third party needs to rerun the tests: the LMCache parallel-read patch (git patch plus full before/after sources), the load client, orchestration scripts, forensic probe scripts and the raw benchmark data.
AMD Instinct MI308X · ROCm 7.2 - R9FX100 on Huawei Ascend 910B: Inference and Training Benchmark (vs. NFS Baseline)Three comparison groups against an NFS baseline on an Ascend platform with 8 × Huawei Atlas 910B: model loading for inference (DeepSeek-32B / 70B), training weight and checkpoint I/O (Qwen-7B), and training-data acceleration (YOLOv8 / COCO). The platform is Ascend 910B, not MI308X, and is labeled as such wherever this site quotes it.
8 × Huawei Atlas 910B (Kunpeng-920)
Related products
Test jointly first, decide afterwards
Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.
G1Week 2
Delivery acceptance
All materials delivered and inspected; every array passes its power-on self-test with all 24 drives recognized.
G2Week 4
Single-node baseline
The single-node 8-GPU inference baseline is reproduced within ±10% of the R1 figures; a local stress test of one array reaches 83% line efficiency against the measured anchor.
G3Week 7
Main joint-test gate
KV tiering must deliver both a TTFT reduction of ≥25% and a throughput gain inside the measured +29–40% band; with 8 nodes sharing one array, read-back bandwidth approaches the NIC ceiling.
G4Week 10
Stability and sign-off
72 hours of continuous load without interruption; single-drive and single-link fault injection stays invisible to the workload; a signed joint-test report is issued.
Other solutions
- Storage Acceleration: KV Cache Tiering
Storage acceleration
- AI Datacenter Efficiency Optimization
Efficiency optimization
- AI Datacenter Construction
Datacenter construction
- Inference-Stack Software Development
Software development
Test jointly first, decide afterwards
Test jointly first, decide afterwards
Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.
- Contact
- Wang Limin
- Phone
- +86 131 6181 8898
- mingxin@agentmail.to
The inbox is monitored and triaged by AI around the clock