LoRA adapter (PEFT) for the Sovereign Judge model in the MoE Sovereign project. Fine-tuned on Qwen/Qwen3-30B-A3B (30B total, ~3B active, 256 experts) for paraconsistent conflict arbitration and quality-assurance evaluation in multi-expert orchestration pipelines.

For the full model description, architecture, output schema, and deployment instructions, see the GGUF repository.

Loading the adapter

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-30B-A3B",
    trust_remote_code=True,
    dtype="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-30B-A3B")
model = PeftModel.from_pretrained(base, "h3rb3rn/Qwen3-MoE-35B-Sovereign-Judge-v3-LoRA")

Related repositories

Repository	Contents
GGUF (this model, quantised)	Q4_K_M and F16 GGUF — recommended for Ollama / llama.cpp
Merged BF16	Full merged model (no adapter loading required)

Training summary

Parameter	Value
Base model	`Qwen/Qwen3-30B-A3B`
LoRA rank / alpha	r = 64, α = 128
Dataset	`paraconsistent_large.jsonl` (90 103 examples, DE/EN, Alpaca format)
Epochs	3
Total steps	2 112
Learning rate	2 × 10⁻⁴
Hardware	8 × AMD MI250X (LUMI-G)
Final train loss	0.3032
Mean token accuracy	86.72 %