You can deploy this model and start earning money today!

RichardErkhov/
bunnycore_-_Gemma-2-2b-All-in-One-exl2

Name: bunnycore_-_Gemma-2-2b-All-in-One-exl2
Availability: InStock

2403.19522

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Gemma-2-2b-All-in-One - EXL2

Model creator: https://huggingface.co/bunnycore/
Original model: https://huggingface.co/bunnycore/Gemma-2-2b-All-in-One/

Available sizes

Branch	Bits	Description
8_0	8.0	Maximum quality that ExLlamaV2 can produce, near unquantized performance.
6_5	6.5	Very similar to 8.0, good tradeoff of size vs performance, recommended.
5_0	5.0	Slightly lower quality vs 6.5, but usable
4_25	4.25	GPTQ equivalent bits per weight, slightly higher quality.
3_5	3.5	Lower quality, only use if you have to.

Download instructions

With git:

git clone --single-branch --branch 6_5 https://huggingface.co/bunnycore_-_Gemma-2-2b-All-in-One-exl2 Gemma-2-2b-All-in-One-6_5

With huggingface hub:

pip3 install huggingface-hub

To download a specific branch, use the --revision parameter. For example, to download the 6.5 bpw branch: Linux:

huggingface-cli download bunnycore_-_Gemma-2-2b-All-in-One-exl2 --revision 6_5 --local-dir Gemma-2-2b-All-in-One-6_5 --local-dir-use-symlinks False

Windows (which apparently doesn't like _ in folders sometimes?):

huggingface-cli download bunnycore_-_Gemma-2-2b-All-in-One-exl2 --revision 6_5 --local-dir Gemma-2-2b-All-in-One-6.5 --local-dir-use-symlinks False

Original model description:

base_model:

cognitivecomputations/dolphin-2.9.4-gemma2-2b
bunnycore/GemmaOmni2-2B
ehristoforu/Gemma2-2b-it-codealpaca
bunnycore/Gemma2-2b-code-lora
bunnycore/Gemma2-2B-mixed
IlyaGusev/gemma-2-2b-it-abliterated library_name: transformers tags:
mergekit
merge

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using IlyaGusev/gemma-2-2b-it-abliterated as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

yaml

models:
  - model: bunnycore/GemmaOmni2-2B
  - model: ehristoforu/Gemma2-2b-it-codealpaca+bunnycore/Gemma2-2b-code-lora
  - model: cognitivecomputations/dolphin-2.9.4-gemma2-2b
  - model: bunnycore/Gemma2-2B-mixed
  - model: IlyaGusev/gemma-2-2b-it-abliterated
merge_method: model_stock
base_model: IlyaGusev/gemma-2-2b-it-abliterated
dtype: bfloat16

RichardErkhov/bunnycore_-_Gemma-2-2b-All-in-One-exl2