You can deploy this model and start earning money today!

RichardErkhov/
wolfram_-_miqu-1-120b-gguf

Name: wolfram_-_miqu-1-120b-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

miqu-1-120b - GGUF

Model creator: https://huggingface.co/wolfram/
Original model: https://huggingface.co/wolfram/miqu-1-120b/

Name	Quant method	Size
miqu-1-120b.Q2_K.gguf	Q2_K	41.15GB
miqu-1-120b.IQ3_XS.gguf	IQ3_XS	45.78GB
miqu-1-120b.IQ3_S.gguf	IQ3_S	48.4GB
miqu-1-120b.Q3_K_S.gguf	Q3_K_S	48.25GB
miqu-1-120b.IQ3_M.gguf	IQ3_M	50.05GB
miqu-1-120b.Q3_K.gguf	Q3_K	53.85GB
miqu-1-120b.Q3_K_M.gguf	Q3_K_M	53.85GB
miqu-1-120b.Q3_K_L.gguf	Q3_K_L	58.68GB
miqu-1-120b.IQ4_XS.gguf	IQ4_XS	60.36GB
miqu-1-120b.Q4_0.gguf	Q4_0	63.1GB
miqu-1-120b.IQ4_NL.gguf	IQ4_NL	63.7GB
miqu-1-120b.Q4_K_S.gguf	Q4_K_S	63.57GB
miqu-1-120b.Q4_K.gguf	Q4_K	67.19GB
miqu-1-120b.Q4_K_M.gguf	Q4_K_M	67.19GB
miqu-1-120b.Q4_1.gguf	Q4_1	70.09GB
miqu-1-120b.Q5_0.gguf	Q5_0	77.08GB
miqu-1-120b.Q5_K_S.gguf	Q5_K_S	77.08GB
miqu-1-120b.Q5_K.gguf	Q5_K	79.18GB
miqu-1-120b.Q5_K_M.gguf	Q5_K_M	79.18GB
miqu-1-120b.Q5_1.gguf	Q5_1	84.06GB
miqu-1-120b.Q6_K.gguf	Q6_K	91.93GB
miqu-1-120b.Q8_0.gguf	Q8_0	119.06GB

Original model description:

base_model:

152334H/miqu-1-70b-sf language:
en
de
fr
es
it library_name: transformers tags:
mergekit
merge license: other

miqu-1-120b

EXL2: 2.4bpw | 2.65bpw | 3.0bpw | 4.0bpw | 5.0bpw
GGUF: Q2_K-Q5_K_M | IQ3_XXS

This is a 120b frankenmerge of miqu-1-70b created by interleaving layers of miqu-1-70b-sf with itself using mergekit.

Inspired by Venus-120b-v1.2, MegaDolphin-120b, and goliath-120b.

Thanks for the support, CopilotKit - the open-source platform for building in-app AI Copilots into any product, with any LLM model. Check out their GitHub.

Thanks for the EXL2 and GGUF quants, Lone Striker!

Also available: miquliz-120b-v2.0 – Miqu's younger, fresher sister; a new and improved Goliath-like merge of Miqu and lzlv.

Review

u/SomeOddCodeGuy wrote on r/LocalLLaMA:

I love this model. It's slow as Christmas but it's SO GOOD. You did great on this.

But this model is close to getting me to shut down my ChatGPT 4 subscription lol. Between it, Deepseek and a couple others, I'm not sure I'll be using ChatGPT much anymore.

Im using the Q8 at 16k, and I can't express how true it remains to its context. I might try to do some testing this weekend, but its great so far.

I've been using your miqu-1 the past two days and its phenomenal. It understands everything I'm saying in ways only ChatGPT did. I've been purposefully getting more and more vague/relaxed in my speaking, and talking about the most inane stuff, and it just follows right along like a person would.

Miqu-1 does ignore instructions a little. I tried to make a more sarcastic/insulting AI assistant to chat with, and specifically told it (multiple times after a few tries) to not apologize to me after, and it wouldn't stop. So if it made a jab like "Wow, great work spelling that word. Quite the whiz kid huh?", making fun of me for misspelling something, it would refuse to not follow up with "Seriously, though, sometimes misspellings happen" lol. But that's the only issue I've had with it.

(Note: All I did was merge this, though, so the credit mostly belongs to Mistral AI (giving proper attribution!) and the creators of mergekit as well as Venus-120b-v1.2 and MegaDolphin-120b who inspired it.)

Model Details

Max Context: 32764 tokens (kept the weird number from the original/base model)
Layers: 140

Prompt template: Mistral

<s>[INST] {prompt} [/INST]

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

152334H/miqu-1-70b-sf

Configuration

The following YAML configuration was used to produce this model:

yaml

dtype: float16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 20]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [10, 30]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [20, 40]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [30, 50]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [40, 60]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [50, 70]
    model: 152334H/miqu-1-70b-sf
- sources:
  - layer_range: [60, 80]
    model: 152334H/miqu-1-70b-sf

Credits & Special Thanks

original (unreleased) model: mistralai (Mistral AI_)
- ⭐⭐⭐ Use their newer, better, official models here! ⭐⭐⭐
leaked model: miqudev/miqu-1-70b
f16 model: 152334H/miqu-1-70b-sf
mergekit: arcee-ai/mergekit: Tools for merging pretrained large language models.
mergekit_config.yml: nsfwthrowitaway69/Venus-120b-v1.2

Support

My Ko-fi page if you'd like to tip me to say thanks or request specific models to be tested or merged with priority. Also consider supporting your favorite model creators, quantizers, or frontend/backend devs if you can afford to do so. They deserve it!

Disclaimer

This model contains leaked weights and due to its content it should not be used by anyone. 😜

But seriously:

License

What I know: Weights produced by a machine are not copyrightable so there is no copyright owner who could grant permission or a license to use, or restrict usage, once you have acquired the files.

Ethics

What I believe: All generative AI, including LLMs, only exists because it is trained mostly on human data (both public domain and copyright-protected, most likely acquired without express consent) and possibly synthetic data (which is ultimately derived from human data, too). It is only fair if something that is based on everyone's knowledge and data is also freely accessible to the public, the actual creators of the underlying content. Fair use, fair AI!

RichardErkhov/wolfram_-_miqu-1-120b-gguf