Crowdsourced benchmarks
Real measured speeds
Tokens per second measured by people running models on their own machines, alongside the calculator estimate. A community median makes the assumptions easier to check.
- Benchmarks
- 3
- Devices
- 2
- Models
- 1
Measured vs estimated
prelim marks one or two reports; three or more confirms the community median. Estimates use 4,096 context tokens, llama.cpp and the default calibration. Reports are grouped by device, model and quant, so runtime, context and run setup may differ.
2 of 2 rows
Sort any column. On smaller screens, scroll the table to see every measurement.
| MacBook Pro / Mac Studio M2 Max (32GB) | Llama 3.1 8B Q4_K_M | 49.3prelim | 48 | +3% | 2 |
| NVIDIA RTX 4090 (24GB) | Llama 3.1 8B Q4_K_M | 130.2prelim | 124 | +5% | 1 |
Most wanted — no benchmarks yet
Own one of these? Your first report helps fill a gap in the measured data.