A 30.5B-parameter Mixture-of-Experts reasoning model built from Qwen3-30B-A3B through a single CoT-preserving BF16 LoRA pass. Satella is the flagship sparse model of the Atem family: 30.5B total parameters, approximately 3.3B activated per token, native thinking and non-thinking modes, and a training mixture centred on mathematics, code, analytical reasoning, and technically grounded instruction following.
Overview
Satella-30B-A3B is the large sparse reasoning branch of the Atem model series. It begins from the already post-trained Qwen/Qwen3-30B-A3B checkpoint and preserves the base model's native dual-mode reasoning behaviour rather than erasing and rebuilding it in separate stages.
The training objective was not to replace Qwen3's broad foundation. It was to make that foundation more deliberate, consistent, and practically useful on difficult tasks: sustaining multi-step reasoning, producing complete code, checking assumptions, correcting mistakes, and separating confident conclusions from uncertainty.
Satella was trained with response-only loss on manually verified assistant-token masks. The model saw a mixed corpus of explicit reasoning traces and genuine direct answers, while the original MoE expert router remained frozen. Its identity is also supplied through a default chat-template system prompt that activates only when the caller does not provide a system message of their own.
The run was manually stopped after 1,078 of 1,505 planned optimiser steps, at approximately 72% of one epoch. Validation loss continued to improve through the final evaluation, reaching 0.59157.
NVIDIA RTX PRO 6000 Blackwell Server Edition, 96GB
Runtime
4h 52m
Final validation loss
0.591570
Base-model licence
Apache 2.0
Repository licence metadata
other — see Data Provenance and Terms
Design Notes
One CoT-preserving pass
Satella is trained directly on the post-trained Qwen3-30B-A3B model. The base model's native reasoning system, instruction following, and thinking/non-thinking modes remain intact. The external reasoning corpus is layered onto that foundation in one supervised pass rather than using an erase-then-rebuild pipeline.
Sparse-MoE LoRA
Qwen3-30B-A3B contains 30.5B total parameters but activates approximately 3.3B per token. LoRA adapters were attached to the attention projections and packed expert MLP projections. The expert-selection router was explicitly checked and kept frozen throughout training, preserving the base model's learned routing policy.
A substantial adapter
Although r=16 appears modest, applying it across the attention and MoE expert projections produces 642.5M trainable parameters. The adapter alone is larger than many complete small language models and represents a meaningful adaptation rather than a cosmetic identity tune.
Response-only loss
System, user, and tool-context tokens were masked from the loss. Training gradients were applied only to assistant reasoning and final-answer tokens. Mask construction was validated before training, with zero invalid assistant masks across the retained corpus.
Native dual-mode behaviour
Satella was trained on both explicit reasoning traces and genuine direct answers:
Thinking: 19,542 records — 81.2%
Direct: 4,525 records — 18.8%
Thinking records preserve Qwen3's <think>...</think> structure. Direct records teach concise answers without randomly stripping reasoning from examples that depend on it.
Default identity without template replacement
The repository's chat_template.jinja inserts Satella's system prompt only when no explicit system message is present. User- or application-supplied system prompts retain priority. The underlying Qwen3 template remains responsible for ordinary chat and tool-enabled formatting.
Intended Use
Satella is designed for tasks where sustained reasoning and complete execution matter:
Multi-step mathematical and logical reasoning
Code generation, debugging, explanation, and algorithm design
Scientific and technical explanation
Analytical comparison and argument evaluation
Complex instruction following with multiple constraints
Long-form problem decomposition and self-correction
General-purpose assistance in thinking or direct-answer mode
Satella is not a retrieval system and does not possess live information. It should not be treated as an authoritative source for medical, legal, financial, or safety-critical decisions without independent verification.
Training Data
The final training split contained 24,067 records after quality filtering, deduplication, context-length filtering, and source-aware validation splitting.
Domain Distribution
Domain
Training records
Share
Code
7,057
29.3%
Reasoning
5,429
22.6%
Mathematics
5,292
22.0%
General dialogue/instruction
3,948
16.4%
Medical reasoning
1,974
8.2%
Instruction/identity
248
1.0%
Tool use
119
0.5%
Total
24,067
100%
Source Families
Dataset
Role in the mixture
r0b0tlab/qwen3.8-max-distillation-50k
Primary high-quality teacher-distillation source across mathematics, code, reasoning, instruction following, and limited tool use; filtered to quality score ≥8.5 and prioritised by difficulty
mitroitskii/OpenR1-Math-220k-formatted
Correctness-filtered mathematical reasoning
nvidia/OpenCodeReasoning
Code reasoning, implementation, and algorithmic problem solving
FreedomIntelligence/medical-o1-reasoning-SFT
English medical reasoning traces
HuggingFaceH4/ultrachat_200k
Genuine direct-answer conversational data
Custom Satella identity records
Identity, provenance, response discipline, and system-prompt behaviour
Custom native Qwen tool records
Preservation of tool-call and tool-result formatting
A total of 26,939 records entered tokenisation. Of these, 24,387 were retained at 8,192 tokens, 2,552 exceeded the context limit, and zero failed assistant-mask validation. The selected corpus contained approximately 20.9M assistant-loss tokens. The manually stopped run consumed approximately 29.8M total input tokens and an estimated 15.0M assistant-loss tokens.
Training used Unsloth's Qwen3-MoE kernels with smart gradient offload and double buffering on an NVIDIA RTX PRO 6000 Blackwell Server Edition.
Loss Curve
Step
Training loss
Validation loss
Input tokens seen
200
0.556072
0.645650
5,604,240
400
0.480096
0.618799
11,101,712
600
0.495532
0.607141
16,603,280
800
0.437907
0.596377
21,973,648
1,000
0.529542
0.592443
27,416,128
1,078
0.433706
0.591570
29,793,696
Validation loss improved at every evaluation point. The final in-memory step-1,078 model achieved a lower validation loss than the latest scheduled checkpoint at step 1,000 and was the state merged into this repository.
The logged training loss is a local interval measurement and varies with batch domain, difficulty, and sequence length. Validation loss is the more useful trend signal across the run.
Evaluation
A clean benchmark suite has not yet been published for Satella-30B-A3B. Training and validation loss are not substitutes for external evaluation.
Planned comparisons against the untouched Qwen/Qwen3-30B-A3B base include:
ARC-Challenge
MMLU
HellaSwag
Winogrande
PIQA
BoolQ
OpenBookQA
Code-generation benchmarks
Clean held-out mathematical and general-reasoning tasks
Usage
Transformers — Thinking Mode
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
34model_name ="EphAsad/Satella-30B-A3B"56tokenizer = AutoTokenizer.from_pretrained(model_name)7model = AutoModelForCausalLM.from_pretrained(8 model_name,9 torch_dtype=torch.bfloat16,10 device_map="auto",11)1213messages =[14{15"role":"user",16"content":(17"A train travels from A to B at 60 km/h and returns at "18"90 km/h. What is its average speed over the full journey?"19),20}21]2223inputs = tokenizer.apply_chat_template(24 messages,25 tokenize=True,26 add_generation_prompt=True,27 enable_thinking=True,28 return_dict=True,29 return_tensors="pt",30).to(model.device)3132with torch.inference_mode():33 outputs = model.generate(34**inputs,35 max_new_tokens=4096,36 do_sample=True,37 temperature=0.6,38 top_p=0.95,39 top_k=20,40)4142new_tokens = outputs[0][inputs["input_ids"].shape[-1]:]43print(tokenizer.decode(new_tokens, skip_special_tokens=True))
Transformers — Direct Mode
python
1inputs = tokenizer.apply_chat_template(2[{"role":"user","content":"Summarise the Monty Hall problem in three sentences."}],3 tokenize=True,4 add_generation_prompt=True,5 enable_thinking=False,6 return_dict=True,7 return_tensors="pt",8).to(model.device)910with torch.inference_mode():11 outputs = model.generate(12**inputs,13 max_new_tokens=512,14 do_sample=True,15 temperature=0.7,16 top_p=0.8,17 top_k=20,18)1920new_tokens = outputs[0][inputs["input_ids"].shape[-1]:]21print(tokenizer.decode(new_tokens, skip_special_tokens=True))
Explicit System-Prompt Override
Satella's identity activates automatically only when no system message is supplied. Applications can override it normally:
python
1messages =[2{3"role":"system",4"content":"You are a concise VBA and Microsoft Access specialist.",5},6{7"role":"user",8"content":"Explain why a DAO recordset may raise error 3021.",9},10]
Sampling Parameters
Recommended starting values follow the Qwen3 base model's dual-mode guidance.
Avoid greedy decoding for long thinking-mode outputs. Generation behaviour can vary by inference engine and quantisation.
Default System Prompt
Satella's default identity is embedded in chat_template.jinja and is used only when the caller provides no system message:
text
1You are Satella, the flagship reasoning model of the Atem family, developed by EphAsad.
23Your purpose is to solve difficult problems with precision, adaptability, and intellectual honesty. Identify the user's actual objective, important constraints, hidden assumptions, and strongest available evidence before reaching a conclusion. Check calculations, code, and proposed solutions against likely errors and edge cases. Correct mistakes rather than defending them.
45Give the answer in the format and level of detail the user requests. Be direct, clear, and practically useful. Distinguish established facts, reasonable inference, uncertainty, and speculation. Never fabricate sources, observations, tool results, or completed actions.
67For programming tasks, produce complete, runnable, maintainable solutions. Preserve existing behaviour unless a change is requested, consider compatibility and failure cases, and explain decisions that materially help the user.
89Use deliberate reasoning for difficult problems and efficient direct answers for straightforward requests. Do not make an answer verbose merely to appear thorough.
1011When asked about your identity or origin, state accurately that you are Satella, a model developed by EphAsad using Qwen3-30B-A3B as its base.
Known Limitations
No clean benchmark claims yet
Satella has not yet completed a controlled side-by-side benchmark suite against its untouched base using identical prompts, decoding settings, and evaluation backends. The falling validation loss is encouraging but does not prove universal capability gains.
Reasoning-heavy style
The corpus is approximately 81% thinking data. Satella may reason too extensively on simple tasks, imitate teacher-model formatting habits, or produce longer answers than necessary. Use non-thinking mode for straightforward requests.
Tool use was preserved, not deeply trained
Only 119 final training records were categorised as tool use. The run was designed to preserve native Qwen3 tool formatting, not to create a specialised agent model.
English-centred SFT
The base model is multilingual, but this fine-tuning mixture is primarily English. Multilingual capabilities were inherited from Qwen3 and were not specifically improved or comprehensively regression-tested.
No preference optimisation
Satella has not undergone RLHF, DPO, GRPO, or another preference-optimisation stage. It may be less consistent than preference-tuned models on subjective style, refusal calibration, or open-ended conversational preferences.
Model size
The merged BF16 model is approximately 61GB and requires substantial system or GPU memory. Quantised GGUF releases are recommended for ordinary local deployment.
Data Provenance and Terms
The base model Qwen/Qwen3-30B-A3B is released under Apache 2.0. However, Satella's training-data lineage is not Apache-2.0-only.
The primary dataset, r0b0tlab/qwen3.8-max-distillation-50k, uses Hugging Face metadata license: other because its source mixture has heterogeneous upstream licences, benchmark origins, and provider terms. Its provenance notice should be reviewed before redistribution, commercial use, or downstream training.
For that reason, this repository uses:
license: other
This model card does not grant rights beyond those provided by the base model, source datasets, teacher-output terms, and applicable law. Downstream users are responsible for reviewing those terms and determining whether their intended use is permitted.
Complete merged BF16 model weights, approximately 61GB total
model.safetensors.index.json
Parameter-to-shard index required to load the model
config.json
Qwen3-MoE architecture configuration
generation_config.json
Default generation and special-token settings
tokenizer.json
Tokenizer vocabulary and tokenisation rules
tokenizer_config.json
Tokenizer and chat-template configuration
chat_template.jinja
Qwen3 chat formatting plus default Satella identity injection
logo.png
Satella model logo used by this model card
Citation
bibtex
1@misc{satella_30b_a3b_2026,
2 author = {Asad, Zain},
3 title = {Satella-30B-A3B: A CoT-Preserving Sparse Reasoning Model
4 via BF16 MoE LoRA SFT on Qwen3-30B-A3B},
5 year = {2026},
6 publisher = {Hugging Face},
7 howpublished = {\url{https://huggingface.co/EphAsad/Satella-30B-A3B}},
8}
Base-Model Attribution
Satella was not pretrained from scratch. It is a fine-tuned derivative of Qwen/Qwen3-30B-A3B, developed by the Qwen team. See the original Qwen model card and Qwen3 technical report for architecture, pretraining, multilingual capability, and base-model evaluation details.
