You can deploy this model and start earning money today!

RichardErkhov/
anthracite-org_-_magnum-32b-v2-gguf

Name: anthracite-org_-_magnum-32b-v2-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

magnum-32b-v2 - GGUF

Model creator: https://huggingface.co/anthracite-org/
Original model: https://huggingface.co/anthracite-org/magnum-32b-v2/

Name	Quant method	Size
magnum-32b-v2.Q2_K.gguf	Q2_K	11.38GB
magnum-32b-v2.IQ3_XS.gguf	IQ3_XS	12.67GB
magnum-32b-v2.IQ3_S.gguf	IQ3_S	13.34GB
magnum-32b-v2.Q3_K_S.gguf	Q3_K_S	13.3GB
magnum-32b-v2.IQ3_M.gguf	IQ3_M	13.69GB
magnum-32b-v2.Q3_K.gguf	Q3_K	14.73GB
magnum-32b-v2.Q3_K_M.gguf	Q3_K_M	14.73GB
magnum-32b-v2.Q3_K_L.gguf	Q3_K_L	15.94GB
magnum-32b-v2.IQ4_XS.gguf	IQ4_XS	16.52GB
magnum-32b-v2.Q4_0.gguf	Q4_0	17.23GB
magnum-32b-v2.IQ4_NL.gguf	IQ4_NL	17.4GB
magnum-32b-v2.Q4_K_S.gguf	Q4_K_S	17.36GB
magnum-32b-v2.Q4_K.gguf	Q4_K	18.35GB
magnum-32b-v2.Q4_K_M.gguf	Q4_K_M	18.35GB
magnum-32b-v2.Q4_1.gguf	Q4_1	19.08GB
magnum-32b-v2.Q5_0.gguf	Q5_0	20.92GB
magnum-32b-v2.Q5_K_S.gguf	Q5_K_S	20.92GB
magnum-32b-v2.Q5_K.gguf	Q5_K	21.5GB
magnum-32b-v2.Q5_K_M.gguf	Q5_K_M	21.5GB
magnum-32b-v2.Q5_1.gguf	Q5_1	22.77GB
magnum-32b-v2.Q6_K.gguf	Q6_K	24.85GB
magnum-32b-v2.Q8_0.gguf	Q8_0	32.18GB

Original model description:

license: other license_name: tongyi-qianwen license_link: https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE language:

en
zh pipeline_tag: text-generation tags:
chat

This is the third in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This model is fine-tuned on top of Qwen1.5 32B.

Prompting

Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:

"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""

Credits

Stheno dataset (filtered)
NobodyExistsOnTheInternet/claude_3.5s_single_turn_unslop_filtered
NobodyExistsOnTheInternet/PhiloGlanSharegpt
NobodyExistsOnTheInternet/Magpie-Reasoning-Medium-Subset
kalomaze/Opus_Instruct_25k
Nopm/Opus_WritingStruct
Gryphe/Sonnet3.5-SlimOrcaDedupCleaned (A ~16k rows subset)

This model has been a team effort, and the credits goes to all members of Anthracite.

Training

The training was done for 2 epochs. We used 8x NVIDIA H100 Tensor Core GPUs for the full-parameter fine-tuning of the model.

Safety

...

RichardErkhov/anthracite-org_-_magnum-32b-v2-gguf