You can deploy this model and start earning money today!

RichardErkhov/
Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2

Name: Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit - EXL2

Model creator: https://huggingface.co/Lekhansh/
Original model: https://huggingface.co/Lekhansh/Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit/

Available sizes

Branch	Bits	Description
8_0	8.0	Maximum quality that ExLlamaV2 can produce, near unquantized performance.
6_5	6.5	Very similar to 8.0, good tradeoff of size vs performance, recommended.
5_0	5.0	Slightly lower quality vs 6.5, but usable
4_25	4.25	GPTQ equivalent bits per weight, slightly higher quality.
3_5	3.5	Lower quality, only use if you have to.

Download instructions

With git:

git clone --single-branch --branch 6_5 https://huggingface.co/Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2 Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-6_5

With huggingface hub:

pip3 install huggingface-hub

To download a specific branch, use the --revision parameter. For example, to download the 6.5 bpw branch: Linux:

huggingface-cli download Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2 --revision 6_5 --local-dir Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-6_5 --local-dir-use-symlinks False

Windows (which apparently doesn't like _ in folders sometimes?):

huggingface-cli download Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2 --revision 6_5 --local-dir Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-6.5 --local-dir-use-symlinks False

Original model description:

base_model: unsloth/llama-3.2-1b-instruct-bnb-4bit tags:

text-generation-inference
transformers
unsloth
llama
trl license: apache-2.0 language:
en

Uploaded model

Developed by: Lekhansh
License: apache-2.0
Finetuned from model : unsloth/llama-3.2-1b-instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

RichardErkhov/Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2

Available sizes

Download instructions

Original model description:

Uploaded model

RichardErkhov/
Lekhansh_-_Llama-3.2-1B-Instruct-corrector-nonjson_overtrained-merged-16bit-exl2