MiniArt 2.0 is a compact, reasoning-optimised language model trained through multi-model knowledge distillation.
It runs entirely on-device with no GPU required.
MiniArt 2.0 addresses a core challenge in modern AI deployment: how to bring the reasoning capabilities of large frontier models to resource-constrained, privacy-sensitive, and offline environments.
Large models like GPT-5.5, Gemini 3.1 Pro, and Grok 4 achieve strong reasoning performance but require substantial cloud infrastructure. MiniArt 2.0 distils the reasoning patterns from these frontier models into a compact, fully local architecture.
Key Design Goals
Goal
Approach
Reasoning capability
Multi-model distillation from 8+ frontier LLMs
On-device deployment
Q4_K_M 4-bit GGUF for llama.cpp/LM Studio/Ollama
Privacy preservation
100% local inference, zero API calls
Instruction following
LoRA fine-tune on diverse instruction-response pairs
Openness
Apache 2.0 — free for commercial use
Why Distillation?
Knowledge distillation transfers the style, structure, and reasoning patterns from teacher models (frontier LLMs) into a student model (MiniArt 2.0). Rather than training from scratch — which requires enormous compute — distillation leverages pre-existing representations and augments them with targeted fine-tuning.
The result is a model that punches above its weight in instruction-following quality and multi-step reasoning compared to models of similar size trained only on web data.
2. Model Files
File
Format
Size
Use Case
miniart-2.0-q4_k_m.gguf
GGUF Q4_K_M
~379 MB
Recommended — LM Studio, Ollama, llama.cpp
miniart-2.0-f16.gguf
GGUF F16
~950 MB
Full precision inference, research
config.json
JSON
<1 KB
Architecture metadata
inference.py
Python
<10 KB
Python inference example
benchmarks.py
Python
<1 KB
Reproduce benchmark results
Recommended: Download miniart-2.0-q4_k_m.gguf for everyday use. Use miniart-2.0-f16.gguf for maximum accuracy with more RAM available.
3. Architecture & Design
MiniArt 2.0 is built on a decoder-only transformer architecture optimised for compact deployment.
Core Architecture
Property
Value
Architecture
Decoder-only Transformer
Hidden Size
896
Attention Heads
14
Key-Value Heads
2 (Grouped Query Attention)
Layers
24
Intermediate Size
4,864
Vocabulary Size
151,936
Context Window
2,048 tokens (fine-tune) / 32,768 (base)
Position Encoding
Rotary Position Embeddings (RoPE)
Attention
Grouped Query Attention (GQA)
Activation
SiLU (Swish)
Normalisation
RMS Norm
Grouped Query Attention (GQA)
MiniArt 2.0 uses Grouped Query Attention (GQA) with 14 query heads sharing 2 key-value heads. This reduces KV cache memory by ~7× compared to standard multi-head attention, enabling longer effective context windows at lower memory cost.
LoRA Adapter
LoRA Parameter
Value
Rank (r)
8
Alpha (α)
16
Dropout
0.05
Scaling Factor (α/r)
2.0
Target Modules
q_proj, v_proj
Trainable Parameters
~1.2M
Base Parameters (frozen)
~494M
Trainable %
~0.24%
4. Training & Fine-Tuning Methodology
Pipeline Overview
┌──────────────────────────────────────────────────────────┐
│ GitHub Actions Runner │
│ 1. Load base model (bf16, 4-bit NF4 QLoRA) │
│ 2. Load Manusagents distillation dataset │
│ 3. Apply LoRA adapters (r=8, α=16) │
│ 4. Run SFTTrainer for 60 gradient steps │
│ 5. Merge LoRA → full model weights │
│ 6. Convert merged model → F16 GGUF │
│ 7. Quantize F16 GGUF → Q4_K_M GGUF │
│ 8. Run lm-eval benchmarks (15 tasks) │
│ 9. Upload artifacts to HuggingFace │
└──────────────────────────────────────────────────────────┘
Training Configuration
Hyperparameter
Value
Optimizer
AdamW (paged)
Learning Rate
2e-4
LR Schedule
Linear with warmup
Warmup Steps
5
Gradient Steps
60
Batch Size
1 (gradient accumulation = 4)
Max Sequence Length
512 tokens
Precision
BF16 + NF4 QLoRA
Gradient Checkpointing
Enabled
5. Dataset
Property
Value
Dataset ID
Manusagents Multi-Model Distillation
Total Samples
600
Source Models
GPT-5.5, Gemini 3.1 Pro, Grok 4, Claude Fable 5, Mythos 5, Qwen 3.7 Max, and more
