You can deploy this model and start earning money today!

RichardErkhov/
MarinaraSpaghetti_-_NemoMix-Unleashed-12B-gguf

Name: MarinaraSpaghetti_-_NemoMix-Unleashed-12B-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

NemoMix-Unleashed-12B - GGUF

Model creator: https://huggingface.co/MarinaraSpaghetti/
Original model: https://huggingface.co/MarinaraSpaghetti/NemoMix-Unleashed-12B/

Name	Quant method	Size
NemoMix-Unleashed-12B.Q2_K.gguf	Q2_K	4.46GB
NemoMix-Unleashed-12B.IQ3_XS.gguf	IQ3_XS	4.94GB
NemoMix-Unleashed-12B.IQ3_S.gguf	IQ3_S	5.18GB
NemoMix-Unleashed-12B.Q3_K_S.gguf	Q3_K_S	5.15GB
NemoMix-Unleashed-12B.IQ3_M.gguf	IQ3_M	5.33GB
NemoMix-Unleashed-12B.Q3_K.gguf	Q3_K	5.67GB
NemoMix-Unleashed-12B.Q3_K_M.gguf	Q3_K_M	5.67GB
NemoMix-Unleashed-12B.Q3_K_L.gguf	Q3_K_L	6.11GB
NemoMix-Unleashed-12B.IQ4_XS.gguf	IQ4_XS	6.33GB
NemoMix-Unleashed-12B.Q4_0.gguf	Q4_0	6.59GB
NemoMix-Unleashed-12B.IQ4_NL.gguf	IQ4_NL	6.65GB
NemoMix-Unleashed-12B.Q4_K_S.gguf	Q4_K_S	6.63GB
NemoMix-Unleashed-12B.Q4_K.gguf	Q4_K	6.96GB
NemoMix-Unleashed-12B.Q4_K_M.gguf	Q4_K_M	6.96GB
NemoMix-Unleashed-12B.Q4_1.gguf	Q4_1	7.26GB
NemoMix-Unleashed-12B.Q5_0.gguf	Q5_0	7.93GB
NemoMix-Unleashed-12B.Q5_K_S.gguf	Q5_K_S	7.93GB
NemoMix-Unleashed-12B.Q5_K.gguf	Q5_K	8.13GB
NemoMix-Unleashed-12B.Q5_K_M.gguf	Q5_K_M	8.13GB
NemoMix-Unleashed-12B.Q5_1.gguf	Q5_1	8.61GB
NemoMix-Unleashed-12B.Q6_K.gguf	Q6_K	9.37GB
NemoMix-Unleashed-12B.Q8_0.gguf	Q8_0	12.13GB

Original model description:

base_model: [] library_name: transformers tags:

mergekit
merge

Information

Details

Okay, I tried really hard to improve my ChatML merges, but that has gone terribly wrong. Everyone is adding special tokens with different IDs so can't even make a proper union tokenizer for them, damn. Not to mention, I made some... interesting discoveres in regards to some models' context lenghts. You can watch the breakdown of how it went down here: https://www.captiongenerator.com/v/2303039/marinaraspaghetti's-merging-experience.

This one feels a bit different to my previous attempts and seems less prone to repetition, especially on higher contexts, which is great for me! I'll probably improve on it even further, but for now, it feels rather nice. Great for RP and storytelling. All credits and thanks go to the amazing MistralAI, Intervitens, Sao10K and Nbeerbower for their amazing models! Plus, special shoutouts to Parasitic Rogue for ideas and Prodeus Unity and Statuo for cool exl2 quants of my previous merges. Cheers to folks over at the Drummer's server! Have a good one, everyone.

Instruct

Sigh, Mistral Instruct, I'm afraid.

<s>[INST] {system} [/INST]{response}</s>[INST] {user's message} [/INST]{response}</s>

Parameters

I recommend running Temperature 1.0-1.25 with 0.1 Top A or 0.01-0.1 Min P, and with 0.8/1.75/2/0 DRY. Also works with lower Temperatures below 1.0. Nothing more needed.

Settings

You can use my exact settings from here (use the ones from the Mistral Base/Customized folder, I also recommend checking the Mistral Improved folder): https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings/tree/main.

GGUF

https://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF

EXL2

https://huggingface.co/Statuo/NemoMix-Unleashed-EXL2-8bpw

NemoMix-Unleashed-12B

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the della_linear merge method using E:\mergekit\mistralaiMistral-Nemo-Base-2407 as a base.

Models Merged

The following models were included in the merge:

E:\mergekit\intervitens_mini-magnum-12b-v1.1
E:\mergekit\nbeerbower_mistral-nemo-bophades-12B
E:\mergekit\Sao10K_MN-12B-Lyra-v1
E:\mergekit\nbeerbower_mistral-nemo-gutenberg-12B
E:\mergekit\mistralaiMistral-Nemo-Instruct-2407

Configuration

The following YAML configuration was used to produce this model:

yaml

models:
  - model: E:\mergekit\mistralaiMistral-Nemo-Instruct-2407
    parameters:
      weight: 0.1
      density: 0.4
  - model: E:\mergekit\nbeerbower_mistral-nemo-bophades-12B
    parameters:
      weight: 0.12
      density: 0.5
  - model: E:\mergekit\nbeerbower_mistral-nemo-gutenberg-12B
    parameters:
      weight: 0.2
      density: 0.6
  - model: E:\mergekit\Sao10K_MN-12B-Lyra-v1
    parameters:
      weight: 0.25
      density: 0.7
  - model: E:\mergekit\intervitens_mini-magnum-12b-v1.1
    parameters:
      weight: 0.33
      density: 0.8
merge_method: della_linear
base_model: E:\mergekit\mistralaiMistral-Nemo-Base-2407
parameters:
  epsilon: 0.05
  lambda: 1
dtype: bfloat16
tokenizer_source: base

Ko-fi

Enjoying what I do? Consider donating here, thank you!

https://ko-fi.com/spicy_marinara

RichardErkhov/MarinaraSpaghetti_-_NemoMix-Unleashed-12B-gguf