# What Is LMCache?

> LMCache is an open-source KV cache library that plugs into vLLM. It moves KV cache that does not fit in GPU memory down to host RAM and external storage and reads it back on a hit. It is a mainstream open-source implementation of KV cache tiering.

URL: https://vuke.top/en/glossary/lmcache · Updated November 12, 2026

LMCache is an open-source KV cache library that plugs into vLLM. It moves KV cache that does not fit in GPU memory down to host RAM and external storage and reads it back on a hit. It is a mainstream open-source implementation of KV cache tiering.

## In the measurements

Xinyuanyi wrote a parallel-read patch for LMCache: with one GPU at concurrency 16 reading cold from disk, TTFT falls from 37.97 s to 9.30 s (4.1×) [R1](https://vuke.top/en/evidence/R1). The patch ships in the R8 code export [R8](https://vuke.top/en/evidence/R8).

Source: [R1](https://vuke.top/en/evidence/R1) [R8](https://vuke.top/en/evidence/R8)