
Base model	Jackrong/Qwen3.5-9B-DeepSeek-V4-Flash
Architecture	Qwen3.5 (hybrid Gated DeltaNet + Gated Attention, `qwen3_5_text`)
Parameters	9B total / 29M trainable (LoRA, 0.53%)
Fine-tuning	QLoRA (4-bit NF4, double quant, BF16 compute)
LoRA config	r=16, α=32, dropout=0.05, all linear layers
Training data	2,893 DeepSeek-R1 reasoning traces (code + math)
Hardware	NVIDIA L4 (24GB, Ada Lovelace) on AWS g6.xlarge
Runtime	SGLang v0.5.15 (RadixAttention + FP8 + multi-LoRA)
License	Apache 2.0

Training Data

This adapter was trained on a curated mix of DeepSeek-R1 chain-of-thought reasoning traces from the open-r1/Mixture-of-Thoughts dataset:

Code subset (3,000 sampled): DeepSeek-R1 reasoning traces on competitive programming problems (Python + C++), sourced from CodeForces
Math subset (2,000 sampled): DeepSeek-R1 reasoning traces on mathematical problems

After length filtering (≤8192 tokens), 2,893 examples were used for training.

Why This Model

The base model (Qwen3.5-9B-DeepSeek-V4-Flash) is already a reasoning model distilled from DeepSeek-V4. We fine-tuned it further to:

Specialize for agentic coding — AWS, Terraform, Docker, Kubernetes, Linux, Python
Reinforce step-by-step reasoning before code generation
Align to the Galaxy Nexus system prompt for consistent agent behavior

Training Results

Metric	Step 1	Step 50	Step 100
Loss	0.8839	0.5212	~0.50
Token accuracy	73.8%	83.2%	~84%
Improvement	—	-41% loss	-43% loss

Loss decreased 43% over 100 steps, with token accuracy climbing from 73.8% to ~84%.

Benchmark Results

Base model (Qwen3.5-9B-DeepSeek-V4-Flash) scored 10/11 (91%) on our agentic-coding benchmark suite covering Python, Docker, Terraform, Kubernetes, Linux, and reasoning tasks.

Serving with SGLang

bash

# Serve base + adapter via SGLang multi-LoRA
python3 -m sglang.launch_server \
  --model-path Qwen3.5-9B-DeepSeek-V4-Flash \
  --enable-lora \
  --max-lora-rank 16 \
  --lora-paths gnx-coder=/path/to/galaxy-coder-reasoner-v1 \
  --quantization fp8 \
  --kv-cache-dtype fp8 \
  --trust-remote-code \
  --mem-fraction-static 0.88

Then call the adapter:

python

import openai
client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="...")
response = client.chat.completions.create(
    model="gnx-coder",  # ← use the adapter name
    messages=[{"role": "user", "content": "Write a Terraform S3 bucket with versioning"}]
)

Limitations

1 epoch only — trained for 100 steps (~1.7 epochs of the dataset). More training would improve quality.
Sequence length 2048 — some long reasoning traces were truncated. Future versions will use 4096+.
No RLHF/DPO — pure SFT. Future versions will add preference optimization.
Qwen3.5 hybrid architecture — training is slower than standard attention models due to new, unoptimized kernels.

Citation

bibtex

@misc{galaxynexus_coder_reasoner_v1,
  title={Galaxy Nexus Coder-Reasoner v1: QLoRA Fine-tune of Qwen3.5-9B for Agentic Coding},
  author={Galaxy Nexus AI},
  year={2026},
  url={https://huggingface.co/GalaxyNexus/galaxy-coder-reasoner-v1}
}

License

Apache 2.0 — commercial use permitted.