Model field guide
Qwen2.5-0.5B-Instruct
42 of 44 hardware configurations have a supported memory fit at one or more recorded quantizations. Runtime compatibility still needs verification.
Total parameters0.49B
Active parameters0.49B
Recorded context limit32,768
Architecturedense
Verified 2026-10-22 · license metadata: apache-2.0 · commercial use: yes (review the terms).
Actual files, not a size shortcut.
| Quantization | Total file size | Source |
|---|---|---|
| Q4_K_M | 0.458 GiB491,400,032 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q5_K_M | 0.486 GiB522,186,592 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q8_0 | 0.629 GiB675,710,816 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| FP16 | 1.179 GiB1,266,425,696 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
The hardware picture.
Q4_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.50 GiB | fits | 124.5–249.1 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.50 GiB | fits | 138.4–276.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.50 GiB | fits | 138.4–276.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.50 GiB | fits | 249.9–499.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.50 GiB | fits | 249.9–499.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.50 GiB | fits | 249.9–499.8 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.50 GiB | fits | 374.4–748.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.50 GiB | fits | 499.8–999.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.50 GiB | fits | 499.8–999.6 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.50 GiB | fits | 976.8–1953.6 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.50 GiB | fits | 976.8–1953.6 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.50 GiB | fits | 976.8–1953.6 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.50 GiB | fits | 124.5–249.1 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.50 GiB | fits | 249.9–499.8 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.50 GiB | fits | 499.8–999.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.50 GiB | fits | 97.7–195.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.50 GiB | fits | 97.7–195.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.50 GiB | fits | 97.7–195.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.50 GiB | fits | 222.2–444.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.50 GiB | fits | 222.2–444.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.50 GiB | fits | 222.2–444.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.50 GiB | fits | 333.7–667.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.50 GiB | fits | 444.4–888.9 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.50 GiB | fits | 444.4–888.9 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.50 GiB | fits | 666.7–1333.3 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.50 GiB | fits | 666.7–1333.3 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.50 GiB | fits | 293.0–586.1 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.50 GiB | fits | 761.9–1523.8 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.50 GiB | fits | 234.4–468.9 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.50 GiB | fits | 410.3–820.5 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.50 GiB | fits | 583.5–1167.0 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.50 GiB | fits | 820.5–1641.0 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.50 GiB | fits | 364.7–729.3 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.50 GiB | fits | 547.0–1094.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.50 GiB | fits | 729.3–1458.7 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.50 GiB | fits | 781.4–1562.9 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.50 GiB | fits | 1458.7–2917.4 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.50 GiB | fits | 1458.7–2917.4 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.50 GiB | fits | 781.4–1562.9 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.50 GiB | fits | 371.2–742.4 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.50 GiB | fits | 208.4–416.8 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.50 GiB | fits | 222.2–444.4 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.50 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.50 GiB | Needs verification | Estimated speed unavailable |
Q5_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.53 GiB | fits | 117.2–234.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.53 GiB | fits | 130.2–260.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.53 GiB | fits | 130.2–260.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.53 GiB | fits | 235.2–470.3 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.53 GiB | fits | 235.2–470.3 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.53 GiB | fits | 235.2–470.3 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.53 GiB | fits | 352.4–704.7 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.53 GiB | fits | 470.3–940.7 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.53 GiB | fits | 470.3–940.7 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.53 GiB | fits | 919.2–1838.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.53 GiB | fits | 919.2–1838.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.53 GiB | fits | 919.2–1838.4 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.53 GiB | fits | 117.2–234.4 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.53 GiB | fits | 235.2–470.3 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.53 GiB | fits | 470.3–940.7 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.53 GiB | fits | 91.9–183.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.53 GiB | fits | 91.9–183.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.53 GiB | fits | 91.9–183.8 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.53 GiB | fits | 209.1–418.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.53 GiB | fits | 209.1–418.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.53 GiB | fits | 209.1–418.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.53 GiB | fits | 314.1–628.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.53 GiB | fits | 418.2–836.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.53 GiB | fits | 418.2–836.5 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.53 GiB | fits | 627.4–1254.7 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.53 GiB | fits | 627.4–1254.7 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.53 GiB | fits | 275.8–551.5 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.53 GiB | fits | 717.0–1434.0 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.53 GiB | fits | 220.6–441.2 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.53 GiB | fits | 386.1–772.1 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.53 GiB | fits | 549.1–1098.2 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.53 GiB | fits | 772.1–1544.3 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.53 GiB | fits | 343.2–686.3 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.53 GiB | fits | 514.8–1029.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.53 GiB | fits | 686.3–1372.7 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.53 GiB | fits | 735.4–1470.7 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.53 GiB | fits | 1372.7–2745.4 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.53 GiB | fits | 1372.7–2745.4 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.53 GiB | fits | 735.4–1470.7 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.53 GiB | fits | 349.3–698.6 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.53 GiB | fits | 196.1–392.2 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.53 GiB | fits | 209.1–418.2 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.53 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.53 GiB | Needs verification | Estimated speed unavailable |
Q8_0: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.68 GiB | fits | 90.6–181.1 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.68 GiB | fits | 100.6–201.3 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.68 GiB | fits | 100.6–201.3 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.68 GiB | fits | 181.7–363.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.68 GiB | fits | 181.7–363.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.68 GiB | fits | 181.7–363.5 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.68 GiB | fits | 272.3–544.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.68 GiB | fits | 363.5–726.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.68 GiB | fits | 363.5–726.9 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.68 GiB | fits | 710.4–1420.7 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.68 GiB | fits | 710.4–1420.7 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.68 GiB | fits | 710.4–1420.7 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.68 GiB | fits | 90.6–181.1 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.68 GiB | fits | 181.7–363.5 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.68 GiB | fits | 363.5–726.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.68 GiB | fits | 71.0–142.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.68 GiB | fits | 71.0–142.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.68 GiB | fits | 71.0–142.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.68 GiB | fits | 161.6–323.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.68 GiB | fits | 161.6–323.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.68 GiB | fits | 161.6–323.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.68 GiB | fits | 242.7–485.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.68 GiB | fits | 323.2–646.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.68 GiB | fits | 323.2–646.4 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.68 GiB | fits | 484.8–969.6 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.68 GiB | fits | 484.8–969.6 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.68 GiB | fits | 213.1–426.2 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.68 GiB | fits | 554.1–1108.2 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.68 GiB | fits | 170.5–341.0 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.68 GiB | fits | 298.4–596.7 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.68 GiB | fits | 424.3–848.6 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.68 GiB | fits | 596.7–1193.4 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.68 GiB | fits | 265.2–530.4 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.68 GiB | fits | 397.8–795.6 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.68 GiB | fits | 530.4–1060.8 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.68 GiB | fits | 568.3–1136.6 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.68 GiB | fits | 1060.8–2121.6 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.68 GiB | fits | 1060.8–2121.6 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.68 GiB | fits | 568.3–1136.6 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.68 GiB | fits | 269.9–539.9 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.68 GiB | fits | 151.5–303.1 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.68 GiB | fits | 161.6–323.2 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.68 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.68 GiB | Needs verification | Estimated speed unavailable |
FP16: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 2.23 GiB | fits | 48.3–96.6 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 2.23 GiB | fits | 53.7–107.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 2.23 GiB | fits | 53.7–107.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 2.23 GiB | fits | 97.0–193.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 2.23 GiB | fits | 97.0–193.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 2.23 GiB | fits | 97.0–193.9 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 2.23 GiB | fits | 145.3–290.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 2.23 GiB | fits | 193.9–387.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.23 GiB | fits | 193.9–387.9 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 2.23 GiB | fits | 379.0–758.0 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.23 GiB | fits | 379.0–758.0 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 2.23 GiB | fits | 379.0–758.0 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 2.23 GiB | fits | 48.3–96.6 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 2.23 GiB | fits | 97.0–193.9 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.23 GiB | fits | 193.9–387.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 2.23 GiB | fits | 37.9–75.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 2.23 GiB | fits | 37.9–75.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 2.23 GiB | fits | 37.9–75.8 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 2.23 GiB | fits | 86.2–172.5 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 2.23 GiB | fits | 86.2–172.5 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 2.23 GiB | fits | 86.2–172.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 2.23 GiB | fits | 129.5–259.0 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 2.23 GiB | fits | 172.5–344.9 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 2.23 GiB | fits | 172.5–344.9 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 2.23 GiB | fits | 258.7–517.4 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.23 GiB | fits | 258.7–517.4 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 2.23 GiB | fits | 113.7–227.4 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 2.23 GiB | fits | 295.6–591.3 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 2.23 GiB | fits | 91.0–181.9 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 2.23 GiB | fits | 159.2–318.4 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 2.23 GiB | fits | 226.4–452.8 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 2.23 GiB | fits | 318.4–636.8 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 2.23 GiB | fits | 141.5–283.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 2.23 GiB | fits | 212.3–424.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 2.23 GiB | fits | 283.0–566.0 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 2.23 GiB | fits | 303.2–606.4 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 2.23 GiB | fits | 566.0–1132.0 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 2.23 GiB | fits | 566.0–1132.0 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 2.23 GiB | fits | 303.2–606.4 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 2.23 GiB | fits | 144.0–288.1 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 2.23 GiB | fits | 80.9–161.7 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 2.23 GiB | fits | 86.2–172.5 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 2.23 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 2.23 GiB | Needs verification | Estimated speed unavailable |
A “fits” result checks estimated memory only. “Needs verification” preserves missing cache, vision, or placement information. All speed figures are estimated from the published bandwidth formula.
Field-level sources and verification dates
- id: Official source · verified 2026-10-22. Lowercase URL slug derived from the upstream repository ID; baseModel retains the exact Hugging Face identifier.
- name: Official source · verified 2026-10-22
- baseModel: Official source · verified 2026-10-22
- downloads: Official source · verified 2026-10-22
- tasks: Official source · verified 2026-10-22. Chat denotes a text-generating model, including base models that may require a chat template or instruction tuning. Code label is inferred from the repository model-family name. Vision uses pipeline metadata or vision_config.
- license: Official source · verified 2026-10-22
- commercialUse: Official source · verified 2026-10-22. Official upstream organization metadata identifies a permissive standard license. Commercial use is permitted subject to the license terms, notices, and applicable restrictions; this is a metadata-based classification, not an individualized legal approval.
- license_url: Official source · verified 2026-10-22. Pinned license file confirmed in repository listing.
- architecture: Official source · verified 2026-10-22. Classification from model_type and expert/state-space configuration fields (qwen2); hybrid classification takes precedence over expert count.
- parametersB: Official source · verified 2026-10-22. Exact safetensors parameter total reported by the Hugging Face API, divided by 1e9.
- activeParametersB: Official source · verified 2026-10-22. Dense model: all parameters are included. This is not a runtime FLOP measurement.
- contextLength: Official source · verified 2026-10-22. Configured maximum token positions; not a guarantee of runtime support or quality at that length. No speculative RoPE scaling applied.
- kvCache: Official source · verified 2026-10-22. Standard attention KV shape from num_hidden_layers, num_key_value_heads, and head_dim or hidden_size / num_attention_heads; FP16 cache bytes are a separate estimator choice.
- licenseStandard: Official source · verified 2026-10-22. Official standard license text supporting the permissive-license classification; model applicability is identified separately by upstream metadata.