You can deploy this model and start earning money today!

RichardErkhov/
Ba2han_-_Tinypus-1.5B-exl2

Name: Ba2han_-_Tinypus-1.5B-exl2
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Tinypus-1.5B - EXL2

Model creator: https://huggingface.co/Ba2han/
Original model: https://huggingface.co/Ba2han/Tinypus-1.5B/

Available sizes

| Branch | Bits | Description | | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ | | 8_0 | 8.0 | Maximum quality that ExLlamaV2 can produce, near unquantized performance. | | 6_5 | 6.5 | Very similar to 8.0, good tradeoff of size vs performance, recommended. | | 5_0 | 5.0 | Slightly lower quality vs 6.5, but usable on 8GB cards. | | 4_25 | 4.25 | GPTQ equivalent bits per weight, slightly higher quality. | | 3_5 | 3.5 | Lower quality, only use if you have to. |

Download instructions

With git:

git clone --single-branch --branch 6_5 https://huggingface.co/Ba2han_-_Tinypus-1.5B-exl2 Tinypus-1.5B-6_5

With huggingface hub:

pip3 install huggingface-hub

To download a specific branch, use the --revision parameter. For example, to download the 6.5 bpw branch: Linux:

huggingface-cli download Ba2han_-_Tinypus-1.5B-exl2 --revision 6_5 --local-dir Tinypus-1.5B-6_5 --local-dir-use-symlinks False

Windows (which apparently doesn't like _ in folders sometimes?):

huggingface-cli download Ba2han_-_Tinypus-1.5B-exl2 --revision 6_5 --local-dir Tinypus-1.5B-6.5 --local-dir-use-symlinks False

Original model description:

license: mit datasets:

garage-bAInd/Open-Platypus pipeline_tag: text-generation

*drumroll please*

Introducing Tinypus!

I passthrough merged base Tiny Llama Chat with itself, then fine-tuned with around 1/3 of Platypus dataset.

Observations:

It's smarter (I think?)
It sometimes throws "### Instruction:" line. This could be due to the platypus dataset, or the fact that I know jackshit about programming. You can add it to "custom stopping strings" in oobaboga.
It may be possible to train very specialized mini experts and merge them???

Template

Same with TinyLlama/TinyLlama-1.1B-Chat-v1.0

Merge details

slices:

sources:
- model: E://text-generation-webui//models//TinyLlama
  
  layer_range: [0, 12]
sources:
- model: E://text-generation-webui//models//TinyLlama
  
  layer_range: [4, 22]

merge_method: passthrough

dtype: bfloat16

QLoRA Details

Chunk Length: 1152 R/A: 64/128 Epoch: 1 q-k-v-o

RichardErkhov/Ba2han_-_Tinypus-1.5B-exl2

Available sizes

Download instructions

Original model description:

RichardErkhov/
Ba2han_-_Tinypus-1.5B-exl2