Quantized to 4bpw and tested using the mlx_lm utility on a 64GiB URAM M1 Max.

See original model for further details.

This is the 9th in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus.

This model is fine-tuned on top of Yi-1.5-34 B-32 K.

Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:

Below are Instruct and Context templates for use within SillyTavern.

In our testing a min_p of 0.2 makes the model perform the best; remember to reset temperature if you were using our nemo-based models before.

We'd like to thank Recursal / Featherless for sponsoring the compute for this train, Featherless has been hosting our Magnum models since the first 72 B and has given thousands of people access to our models and helped us grow.

We would also like to thank all members of Anthracite who made this finetune possible.

The training was done for 2 epochs. We used 8xH100s GPUs graciously provided by Recursal AI / Featherless AI for the full-parameter fine-tuning of the model.

...

Detailed results can be found here