Solutions
Inference-Stack Software Development
Software development
Updated
The problem
Once the hardware is in place, software is usually what stalls the project: the upstream framework does not support your card, the cache library has a defect under your load, or nobody has adapted a new model type.
Our approach
We change the inference stack at source level and hand the changes over as they are: the patch, the orchestration scripts and the raw data are all in R8R8.
Key points and sources
- Upstream-grade engineering already demonstrated: ROCm builds of vLLM / LMCache, the parallel-read patch, a GDS adaptation layer and fs:// defect isolationR1R8
- Fast validation of new requirements: AI video-generation pipelines (ComfyUI / LTX-Video already running)R6R7, private inference appliances and more
- Developer resources: about 10,000 experienced software developers (the company's own figure, open to due-diligence checks and not used as evidence)
- University cooperation: a partnership with Beijing Information Science and Technology University lets student developers, on the order of ten thousand by the university's own figure, join the ROCm ecosystem through joint courses and open-source practice
Related test reports
- R1FX100 Comprehensive LLM Inference and Training Benchmark (8 × AMD MI308X)One report, five questions: how much the LMCache parallel-read patch helps, whether KV tiering scales across GPUs, how much faster 8 GPUs load models concurrently, how effective utilization changes under frequent model switching, and how much faster concurrent training checkpoint writes become. Test models are Qwen2.5-32B and 7B.
8 × AMD Instinct MI308X · ROCm 7.2 - R6ComfyUI + LTX-Video 2.3 Full-Model Deployment, Adaptation and Usage Report (V2)Proof of adaptation for video generation: all 7 delivered models (3 VAEs, the fp8 text encoder, the text projection and 2 LoRAs) run end to end on MI308X / ROCm 7.2 with no dtype, operator or kernel errors.
AMD Instinct MI308X · ROCm 7.2 - R7ComfyUI + LTX-Video 2.3 Model Adaptation Report (AMD MI308X)The per-model detail behind R6: individual run records for the 3 VAEs and the Singularity LoRA, plus the adaptation points that matter on MI308X.
AMD Instinct MI308X - R8FX100 KV Cache Test Code Export and Raw Benchmark DataEverything a third party needs to rerun the tests: the LMCache parallel-read patch (git patch plus full before/after sources), the load client, orchestration scripts, forensic probe scripts and the raw benchmark data.
AMD Instinct MI308X · ROCm 7.2
Test jointly first, decide afterwards
Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.
G1Week 2
Delivery acceptance
All materials delivered and inspected; every array passes its power-on self-test with all 24 drives recognized.
G2Week 4
Single-node baseline
The single-node 8-GPU inference baseline is reproduced within ±10% of the R1 figures; a local stress test of one array reaches 83% line efficiency against the measured anchor.
G3Week 7
Main joint-test gate
KV tiering must deliver both a TTFT reduction of ≥25% and a throughput gain inside the measured +29–40% band; with 8 nodes sharing one array, read-back bandwidth approaches the NIC ceiling.
G4Week 10
Stability and sign-off
72 hours of continuous load without interruption; single-drive and single-link fault injection stays invisible to the workload; a signed joint-test report is issued.
Other solutions
- Storage Acceleration: KV Cache Tiering
Storage acceleration
- Enablement for Domestic and Non-NVIDIA Accelerators
Accelerator enablement
- AI Datacenter Efficiency Optimization
Efficiency optimization
- AI Datacenter Construction
Datacenter construction
Test jointly first, decide afterwards
Test jointly first, decide afterwards
Every engagement starts with a gated joint test of about 10 weeks. Four gates are passed in sequence; if one is missed, the project stops there, so you never pay for a benefit that was not demonstrated.
- Contact
- Wang Limin
- Phone
- +86 131 6181 8898
- mingxin@agentmail.to
The inbox is monitored and triaged by AI around the clock