The 2.25 bpw quant will fit in three 24 GB cards with 20k tokens of fp16 context.
The 4.00 bpw quant will fit in six 24 GB cards with 81,920 tokens of fp16 context.
Note that all these numbers are on the current version of exllamav3, which does not support tensor parallelism at this time. If you're reading this from a future where this feature has been implemented, or if you have larger cards, then you can probably do better than what I'm reporting here.
