Qwen3-Pinion is a standalone merged checkpoint derived from Qwen/Qwen3-1.7B. The training path was LoRA-based SFT, then adapter merge into full weights, so inference does not require a separate LoRA adapter.

Model DOI: 10.57967/hf/7965
DOI URL: https://doi.org/10.57967/hf/7965

[!WARNING] This checkpoint is SFT-only and has not been re-aligned with DPO/RLHF after SFT. Safety behavior is weaker than the base model. Treat this as a research artifact, not a production safety-aligned assistant.

Why This Release Exists

Single-file loading posture for Transformers users (no base+adapter pairing).
CPU-first friendly model line with downstream GGUF conversions.
Open model artifact release posture, with explicit provenance and licensing boundaries.

Model Details

Property	Value
Base model	Qwen/Qwen3-1.7B
Architecture	`Qwen3ForCausalLM`
Parameters	2,031,739,904
Hidden size	2048
Layers	28
Attention heads	16
KV heads	8
Intermediate size	6144
Max position embeddings	40,960
Vocab size	151,936
Saved dtype	float32
Files	8 safetensor shards + tokenizer/config artifacts

Training and Merge Lineage

Release lineage:

Start from Qwen/Qwen3-1.7B.
Run supervised fine-tuning with LoRA.
Merge adapter deltas into standalone full weights.
Publish merged Transformers checkpoint as canonical artifact.

The direct merge operation for this model line is PEFT merge_and_unload() into safetensors.

Quick Start (Transformers, CPU)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Somnus-Sovereign-Systems/qwen3-pinion"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    device_map="cpu",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain gradient descent in plain language."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.6,
        top_k=20,
        top_p=0.95,
        do_sample=True,
    )

print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

Evaluation Status

Early local CPU smoke tests indicate stronger instruction-following behavior versus base in short interactive prompts, but this is not a formal benchmark suite. Treat quality claims as preliminary unless backed by reproducible eval tables.

Safety and Limitations

Safety alignment was not restored after SFT.
This release is pre-DPO/pre-RL preference optimization.
Hallucinations and over-compliance remain possible.
Not suitable for safety-critical, legal, medical, or financial deployment.

Licensing and Provenance Boundary

License

This repository distributes the Qwen3-Pinion Open Weights artifacts only.

The model weights, tokenizer files, configuration files, and related artifacts in this repository are released under the Qwen3-Pinion Open Weights Notice.

The training and pipeline code used to produce these artifacts is licensed separately under GNU GPL v3.0 (GPLv3) in its respective code repository:

Reinforcement-Learning-Full-Pipeline

Users must also comply with the applicable upstream terms for:

Qwen/Qwen3-1.7B
Magpie-Align/Magpie-Pro-300K-Filtered

Intended Use

research and experimentation
local inference and comparative model analysis
downstream quantization and deployment prototyping

Out-of-Scope Use

production deployment without additional alignment and safety controls
harmful, deceptive, or abusive usage