You can deploy this model and start earning money today!

ParrotRouter/
Qwen3-4B-Thinking-2507-20250813-033307-1

Name: Qwen3-4B-Thinking-2507-20250813-033307-1
Availability: InStock

Views

No views yet

Qwen3-4B Merged - Configuration 1

This is a Qwen3-4B based model created through layer-wise merging of multiple fine-tuned variants to optimize performance on GPQA Diamond.

Performance Metrics

Benchmark	Score	Description
MMLU (5-shot)	0.6837 (68.37%)	Massive Multitask Language Understanding
GPQA Diamond (0-shot)	0.4343 (43.43%)	Graduate-level Physics Q&A

Benchmark Details

MMLU: Evaluated on the test set with 5-shot prompting across 57 subjects
GPQA: Evaluated on the diamond subset with 0-shot prompting on graduate-level physics questions

Performance Visualizations

GPQA Diamond Performance Comparison

MMLU and GPQA Diamond Combined Performance

Model Information

Run ID: 20250813_033307
Optimization Task: GPQA (Graduate-level Physics Q&A)
Number of Layers: 36
Base Architecture: Qwen3-4B

Source Models

The following models were used in the layer-wise merge:

Qwen/Qwen3-4B-Thinking-2507
Qwen/Qwen3-4B-Thinking-2507-FP8
unsloth/Qwen3-4B-Thinking-2507
ertghiu256/Qwen3-4b-tcomanr-merge-v2
huihui-ai/Huihui-Qwen3-4B-Thinking-2507-abliterated
janhq/Jan-v1-4B
BRlkl/BingoGuard-qwen3-4B-pt-grpo
fireworks1231/agentic-4b-2607-sft
sequelbox/Qwen3-4B-Thinking-2507-DAG-Reasoning
ReallyFloppyPenguin/Mastermind-2x4b-thinking

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model
model = AutoModelForCausalLM.from_pretrained(
    "EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1")

# Example: MMLU-style question
prompt = '''Question: The study of the distribution and determinants of health and disease in populations is:
A) Epidemiology
B) Ecology  
C) Etiology
D) Endocrinology
Answer:'''

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_length=150,
    temperature=0.7,
    do_sample=True
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Inference with vLLM

python

from vllm import LLM, SamplingParams

llm = LLM(model="EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1")
sampling_params = SamplingParams(temperature=0.7, top_p=0.95, max_tokens=256)

prompts = ["Question: Explain quantum entanglement in simple terms."]
outputs = llm.generate(prompts, sampling_params)

Technical Details

This model uses a layer-wise merging approach where each transformer layer is selected from different source models based on optimization criteria. This technique allows combining strengths from multiple fine-tuned models.

Merging Process

Layer Selection: Each layer (0-35 for this architecture) is independently selected from one of the source models
Non-layer Weights: Embeddings and final layers are taken from the base model
Optimization: The configuration was found through systematic optimization on the target benchmark

Limitations

This is an experimental merge and performance may vary on tasks outside the optimization targets
The model inherits limitations from its source models
Performance on general tasks may differ from benchmark scores

Citation

If you use this model, please cite the original source models and egan.ai

Note

This model is provided for research purposes. Always validate performance on your specific use case before deployment.

ParrotRouter/Qwen3-4B-Thinking-2507-20250813-033307-1