Model field guide
granite-3.3-2b-instruct
42 of 44 hardware configurations have a supported memory fit at one or more recorded quantizations. Runtime compatibility still needs verification.
Total parameters2.53B
Active parameters2.53B
Recorded context limit131,072
Architecturedense
Verified 2026-10-22 · license metadata: apache-2.0 · commercial use: yes (review the terms).
Actual files, not a size shortcut.
| Quantization | Total file size | Source |
|---|---|---|
| Q4_K_M | 1.439 GiB1,545,303,328 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q5_K_M | 1.681 GiB1,804,825,888 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q8_0 | 2.509 GiB2,694,120,096 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| FP16 | 4.721 GiB5,069,158,176 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
The hardware picture.
Q4_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 2.75 GiB | fits | 39.6–79.2 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 2.75 GiB | fits | 44.0–88.0 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 2.75 GiB | fits | 44.0–88.0 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 2.75 GiB | fits | 79.5–158.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 2.75 GiB | fits | 79.5–158.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 2.75 GiB | fits | 79.5–158.9 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 2.75 GiB | fits | 119.1–238.1 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 2.75 GiB | fits | 158.9–317.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.75 GiB | fits | 158.9–317.9 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 2.75 GiB | fits | 310.6–621.2 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.75 GiB | fits | 310.6–621.2 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 2.75 GiB | fits | 310.6–621.2 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 2.75 GiB | fits | 39.6–79.2 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 2.75 GiB | fits | 79.5–158.9 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.75 GiB | fits | 158.9–317.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 2.75 GiB | fits | 31.1–62.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 2.75 GiB | fits | 31.1–62.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 2.75 GiB | fits | 31.1–62.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 2.75 GiB | fits | 70.7–141.3 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 2.75 GiB | fits | 70.7–141.3 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 2.75 GiB | fits | 70.7–141.3 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 2.75 GiB | fits | 106.1–212.3 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 2.75 GiB | fits | 141.3–282.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 2.75 GiB | fits | 141.3–282.7 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 2.75 GiB | fits | 212.0–424.0 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.75 GiB | fits | 212.0–424.0 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 2.75 GiB | fits | 93.2–186.4 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 2.75 GiB | fits | 242.3–484.6 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 2.75 GiB | fits | 74.5–149.1 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 2.75 GiB | fits | 130.5–260.9 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 2.75 GiB | fits | 185.5–371.1 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 2.75 GiB | fits | 260.9–521.8 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 2.75 GiB | fits | 116.0–231.9 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 2.75 GiB | fits | 173.9–347.9 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 2.75 GiB | fits | 231.9–463.9 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 2.75 GiB | fits | 248.5–497.0 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 2.75 GiB | fits | 463.9–927.7 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 2.75 GiB | fits | 463.9–927.7 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 2.75 GiB | fits | 248.5–497.0 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 2.75 GiB | fits | 118.0–236.1 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 2.75 GiB | fits | 66.3–132.5 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 2.75 GiB | fits | 70.7–141.3 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 2.75 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 2.75 GiB | Needs verification | Estimated speed unavailable |
Q5_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 2.99 GiB | fits | 33.9–67.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 2.99 GiB | fits | 37.7–75.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 2.99 GiB | fits | 37.7–75.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 2.99 GiB | fits | 68.0–136.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 2.99 GiB | fits | 68.0–136.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 2.99 GiB | fits | 68.0–136.1 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 2.99 GiB | fits | 101.9–203.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 2.99 GiB | fits | 136.1–272.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.99 GiB | fits | 136.1–272.2 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 2.99 GiB | fits | 266.0–531.9 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.99 GiB | fits | 266.0–531.9 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 2.99 GiB | fits | 266.0–531.9 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 2.99 GiB | fits | 33.9–67.8 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 2.99 GiB | fits | 68.0–136.1 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 2.99 GiB | fits | 136.1–272.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 2.99 GiB | fits | 26.6–53.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 2.99 GiB | fits | 26.6–53.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 2.99 GiB | fits | 26.6–53.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 2.99 GiB | fits | 60.5–121.0 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 2.99 GiB | fits | 60.5–121.0 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 2.99 GiB | fits | 60.5–121.0 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 2.99 GiB | fits | 90.9–181.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 2.99 GiB | fits | 121.0–242.0 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 2.99 GiB | fits | 121.0–242.0 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 2.99 GiB | fits | 181.5–363.0 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 2.99 GiB | fits | 181.5–363.0 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 2.99 GiB | fits | 79.8–159.6 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 2.99 GiB | fits | 207.4–414.9 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 2.99 GiB | fits | 63.8–127.7 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 2.99 GiB | fits | 111.7–223.4 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 2.99 GiB | fits | 158.9–317.7 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 2.99 GiB | fits | 223.4–446.8 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 2.99 GiB | fits | 99.3–198.6 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 2.99 GiB | fits | 148.9–297.9 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 2.99 GiB | fits | 198.6–397.2 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 2.99 GiB | fits | 212.8–425.5 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 2.99 GiB | fits | 397.2–794.3 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 2.99 GiB | fits | 397.2–794.3 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 2.99 GiB | fits | 212.8–425.5 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 2.99 GiB | fits | 101.1–202.1 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 2.99 GiB | fits | 56.7–113.5 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 2.99 GiB | fits | 60.5–121.0 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 2.99 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 2.99 GiB | Needs verification | Estimated speed unavailable |
Q8_0: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 3.82 GiB | fits | 22.7–45.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 3.82 GiB | fits | 25.2–50.5 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 3.82 GiB | fits | 25.2–50.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 3.82 GiB | fits | 45.6–91.2 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 3.82 GiB | fits | 45.6–91.2 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 3.82 GiB | fits | 45.6–91.2 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 3.82 GiB | fits | 68.3–136.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 3.82 GiB | fits | 91.2–182.3 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.82 GiB | fits | 91.2–182.3 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 3.82 GiB | fits | 178.2–356.3 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.82 GiB | fits | 178.2–356.3 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 3.82 GiB | fits | 178.2–356.3 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 3.82 GiB | fits | 22.7–45.4 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 3.82 GiB | fits | 45.6–91.2 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.82 GiB | fits | 91.2–182.3 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 3.82 GiB | fits | 17.8–35.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 3.82 GiB | fits | 17.8–35.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 3.82 GiB | fits | 17.8–35.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 3.82 GiB | fits | 40.5–81.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 3.82 GiB | fits | 40.5–81.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 3.82 GiB | fits | 40.5–81.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 3.82 GiB | fits | 60.9–121.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 3.82 GiB | fits | 81.1–162.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 3.82 GiB | fits | 81.1–162.1 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 3.82 GiB | fits | 121.6–243.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.82 GiB | fits | 121.6–243.2 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 3.82 GiB | fits | 53.4–106.9 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 3.82 GiB | fits | 139.0–277.9 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 3.82 GiB | fits | 42.8–85.5 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 3.82 GiB | fits | 74.8–149.7 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 3.82 GiB | fits | 106.4–212.8 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 3.82 GiB | fits | 149.7–299.3 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 3.82 GiB | fits | 66.5–133.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 3.82 GiB | fits | 99.8–199.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 3.82 GiB | fits | 133.0–266.1 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 3.82 GiB | fits | 142.5–285.1 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 3.82 GiB | fits | 266.1–532.1 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 3.82 GiB | fits | 266.1–532.1 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 3.82 GiB | fits | 142.5–285.1 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 3.82 GiB | fits | 67.7–135.4 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 3.82 GiB | fits | 38.0–76.0 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 3.82 GiB | fits | 40.5–81.1 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 3.82 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 3.82 GiB | Needs verification | Estimated speed unavailable |
FP16: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 6.03 GiB | fits | 12.1–24.1 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 6.03 GiB | fits | 13.4–26.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 6.03 GiB | fits | 13.4–26.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 6.03 GiB | fits | 24.2–48.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 6.03 GiB | fits | 24.2–48.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 6.03 GiB | fits | 24.2–48.4 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 6.03 GiB | fits | 36.3–72.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 6.03 GiB | fits | 48.4–96.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.03 GiB | fits | 48.4–96.9 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 6.03 GiB | fits | 94.7–189.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.03 GiB | fits | 94.7–189.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 6.03 GiB | fits | 94.7–189.4 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 6.03 GiB | fits | 12.1–24.1 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 6.03 GiB | fits | 24.2–48.4 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.03 GiB | fits | 48.4–96.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 6.03 GiB | fits | 9.5–18.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 6.03 GiB | fits | 9.5–18.9 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 6.03 GiB | fits | 9.5–18.9 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 6.03 GiB | fits | 21.5–43.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 6.03 GiB | fits | 21.5–43.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 6.03 GiB | fits | 21.5–43.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 6.03 GiB | fits | 32.4–64.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 6.03 GiB | fits | 43.1–86.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 6.03 GiB | fits | 43.1–86.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 6.03 GiB | fits | 64.6–129.3 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.03 GiB | fits | 64.6–129.3 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 6.03 GiB | fits | 28.4–56.8 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 6.03 GiB | fits | 73.9–147.7 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 6.03 GiB | fits | 22.7–45.5 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 6.03 GiB | fits | 39.8–79.5 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 6.03 GiB | fits | 56.6–113.1 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 6.03 GiB | fits | 79.5–159.1 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 6.03 GiB | fits | 35.4–70.7 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 6.03 GiB | fits | 53.0–106.1 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 6.03 GiB | fits | 70.7–141.4 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 6.03 GiB | fits | 75.8–151.5 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 6.03 GiB | fits | 141.4–282.8 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 6.03 GiB | fits | 141.4–282.8 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 6.03 GiB | fits | 75.8–151.5 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 6.03 GiB | fits | 36.0–72.0 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 6.03 GiB | fits | 20.2–40.4 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 6.03 GiB | fits | 21.5–43.1 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 6.03 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 6.03 GiB | Needs verification | Estimated speed unavailable |
A “fits” result checks estimated memory only. “Needs verification” preserves missing cache, vision, or placement information. All speed figures are estimated from the published bandwidth formula.
Field-level sources and verification dates
- id: Official source · verified 2026-10-22. Lowercase URL slug derived from the upstream repository ID; baseModel retains the exact Hugging Face identifier.
- name: Official source · verified 2026-10-22
- baseModel: Official source · verified 2026-10-22
- downloads: Official source · verified 2026-10-22
- tasks: Official source · verified 2026-10-22. Chat denotes a text-generating model, including base models that may require a chat template or instruction tuning. Code label is inferred from the repository model-family name. Vision uses pipeline metadata or vision_config.
- license: Official source · verified 2026-10-22
- commercialUse: Official source · verified 2026-10-22. Official upstream organization metadata identifies a permissive standard license. Commercial use is permitted subject to the license terms, notices, and applicable restrictions; this is a metadata-based classification, not an individualized legal approval.
- license_url: Official source · verified 2026-10-22. No recognized standalone license file; linked pinned model card.
- architecture: Official source · verified 2026-10-22. Classification from model_type and expert/state-space configuration fields (granite); hybrid classification takes precedence over expert count.
- parametersB: Official source · verified 2026-10-22. Exact safetensors parameter total reported by the Hugging Face API, divided by 1e9.
- activeParametersB: Official source · verified 2026-10-22. Dense model: all parameters are included. This is not a runtime FLOP measurement.
- contextLength: Official source · verified 2026-10-22. Configured maximum token positions; not a guarantee of runtime support or quality at that length. No speculative RoPE scaling applied.
- kvCache: Official source · verified 2026-10-22. Standard attention KV shape from num_hidden_layers, num_key_value_heads, and head_dim or hidden_size / num_attention_heads; FP16 cache bytes are a separate estimator choice.
- licenseStandard: Official source · verified 2026-10-22. Official standard license text supporting the permissive-license classification; model applicability is identified separately by upstream metadata.