This was made to test extremely low quants. 70B is almost usable at 1.45 bpw. A custom strategy for quanting was used, for details check EXL3 repo. This is borderline usable, the goal was to run 70b model under 16gb of vram. If you have 16GB VRAM use 4k context and it will fit, and sort of work. very borderline, but can be used.