You can deploy this model and start earning money today!

RichardErkhov/
ludocomito_-_Minerva-MoE-2x3B-gguf

Name: ludocomito_-_Minerva-MoE-2x3B-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Minerva-MoE-2x3B - GGUF

Model creator: https://huggingface.co/ludocomito/
Original model: https://huggingface.co/ludocomito/Minerva-MoE-2x3B/

Name	Quant method	Size
Minerva-MoE-2x3B.Q2_K.gguf	Q2_K	1.77GB
Minerva-MoE-2x3B.Q3_K_S.gguf	Q3_K_S	2.07GB
Minerva-MoE-2x3B.Q3_K.gguf	Q3_K	2.3GB
Minerva-MoE-2x3B.Q3_K_M.gguf	Q3_K_M	2.3GB
Minerva-MoE-2x3B.Q3_K_L.gguf	Q3_K_L	2.49GB
Minerva-MoE-2x3B.IQ4_XS.gguf	IQ4_XS	2.58GB
Minerva-MoE-2x3B.Q4_0.gguf	Q4_0	2.69GB
Minerva-MoE-2x3B.IQ4_NL.gguf	IQ4_NL	2.72GB
Minerva-MoE-2x3B.Q4_K_S.gguf	Q4_K_S	2.71GB
Minerva-MoE-2x3B.Q4_K.gguf	Q4_K	2.87GB
Minerva-MoE-2x3B.Q4_K_M.gguf	Q4_K_M	2.87GB
Minerva-MoE-2x3B.Q4_1.gguf	Q4_1	2.98GB
Minerva-MoE-2x3B.Q5_0.gguf	Q5_0	3.28GB
Minerva-MoE-2x3B.Q5_K_S.gguf	Q5_K_S	3.28GB
Minerva-MoE-2x3B.Q5_K.gguf	Q5_K	3.37GB
Minerva-MoE-2x3B.Q5_K_M.gguf	Q5_K_M	3.37GB
Minerva-MoE-2x3B.Q5_1.gguf	Q5_1	3.57GB
Minerva-MoE-2x3B.Q6_K.gguf	Q6_K	3.9GB
Minerva-MoE-2x3B.Q8_0.gguf	Q8_0	5.04GB

Original model description:

license: apache-2.0 tags:

moe
frankenmoe
merge
mergekit
lazymergekit
DeepMount00/Minerva-3B-base-RAG
FairMind/Minerva-3B-Instruct-v1.0 base_model:
DeepMount00/Minerva-3B-base-RAG
FairMind/Minerva-3B-Instruct-v1.0

Minerva-MoE-3x3B

Minerva-MoE-3x3B is a Mixture of Experts (MoE) made with the following models using LazyMergekit:

Evaluation

arc_it acc_norm: 31.91 hellaswag_it acc_norm: 52.20 mmmlu_it: 25.72

🧩 Configuration

yaml

base_model: sapienzanlp/Minerva-3B-base-v1.0
experts:
  - source_model: DeepMount00/Minerva-3B-base-RAG
    positive_prompts:
    - "rispondi a domande"
    - "cosa è"
    - "chi è"
    - "dove è"
    - "come si"
    - "spiegami"
    - "definisci"
  - source_model: FairMind/Minerva-3B-Instruct-v1.0
    positive_prompts:
    - "istruzione"
    - "input"
    - "risposta"
    - "scrivi"
    - "sequenza"
    - "istruzioni"
dtype: bfloat16

💻 Usage

python

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "ludocomito/Minerva-MoE-3x3B"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

RichardErkhov/ludocomito_-_Minerva-MoE-2x3B-gguf