This is a 4.6bpw EXL2 quant of anthracite-org/magnum-v3-34b

This quant was made using exllamav2-0.1.9 with default dataset.

I tested this quant shortly in some random RPs (including 8k+ RPs where remembering and understanding specific facts in the context is needed) and it seems to work fine (in my own testing it is much better than official exl2 4.0bpw quant).

Uses ChatML format like mentioned below.

This is the 9th in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus.

This model is fine-tuned on top of Yi-1.5-34 B-32 K.

Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:

Below are Instruct and Context templates for use within SillyTavern.

In our testing a min_p of 0.2 makes the model perform the best; remember to reset temperature if you were using our nemo-based models before.

We'd like to thank Recursal / Featherless for sponsoring the compute for this train, Featherless has been hosting our Magnum models since the first 72 B and has given thousands of people access to our models and helped us grow.

We would also like to thank all members of Anthracite who made this finetune possible.

The training was done for 2 epochs. We used 8xH100s GPUs graciously provided by Recursal AI / Featherless AI for the full-parameter fine-tuning of the model.

...