It contains several quantizations, each in its own branch:
main (q4f16_1) <-- You are currently on this branch
q4f16_2
q8f16_1
autogptq_llama_q4f16_1
To run the model, please check out the MLC instructions.
In case the model libraries are not yet available in the binary lib srepo, please obtain them from this PR
