# What Is TP (tensor parallelism)?

> Tensor parallelism (TP) splits every layer of a model across several GPUs that compute it together. TP=8 means one model instance spans 8 GPUs, a common way to deploy models too large for a single card; TP4×2 divides 8 GPUs into two instances of 4 GPUs each.

URL: https://vuke.top/en/glossary/tensor-parallelism · Updated November 12, 2026

Tensor parallelism (TP) splits every layer of a model across several GPUs that compute it together. TP=8 means one model instance spans 8 GPUs, a common way to deploy models too large for a single card; TP4×2 divides 8 GPUs into two instances of 4 GPUs each.

## In the measurements

R2 uses a single TP=8 instance and R3 uses two TP4 instances; KV tiering is effective in both layouts [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3).

Source: [R2](https://vuke.top/en/evidence/R2) [R3](https://vuke.top/en/evidence/R3)