The adapter is intended to reinforce practical instruction following, complete technical answers, code-oriented assistance, RAG-driven workflows, and context-aware translation/localization. It is not a full standalone model checkpoint. It must be loaded together with the base model.

Status

This release is an adapter release with training metadata and logs available locally from the training package. It is not yet a benchmarked research release.

What is documented:

Training data mix and example counts
Approximate training token count from trainer logs
LoRA/QLoRA hyperparameters
Training steps, runtime, checkpoints, and final training loss
Reproducible training script structure

What is not yet proven by this release:

It has not yet been independently benchmarked against the base model.
It has not yet been evaluated on HumanEval, MBPP, SWE-bench, translation benchmarks, or general capability regression tests.
It should not be claimed to be globally better than Qwen/Qwen3.5-9B-Base until a separate evaluation report is added.

Model Details

Model name: Qwen Existence Code
Repository name: Qwen3.5-9B-Existence-Code
Base model: Qwen/Qwen3.5-9B-Base
Model type: PEFT LoRA adapter
Training method: QLoRA SFT
Adapter file: adapter_model.safetensors
Adapter size: about 58 MB
License: Apache-2.0
Primary languages: English and Russian

Files

text

README.md
adapter_model.safetensors
adapter_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
training_args.bin

Intended Use

Typical use cases:

Code generation and code completion
Refactoring and debugging assistance
Technical documentation
RAG-based project assistants
Automation planning
Game localization and dialogue translation
Context-aware translation where tone, slang, mature language, or character voice must be preserved

For localization tasks, the adapter is intended to preserve the meaning and tone of source material when the user has the right to process that content. Fictional game dialogue may include profanity, slang, dark humor, or mature language, and removing it can damage the original intent.

Training Data

The training set was built as a mixed SFT JSONL file:

data/sft_mix.jsonl

Observed dataset statistics:

Source	Examples
`microsoft/orca-agentinstruct-1M-v1`	500
`HuggingFaceH4/ultrafeedback_binarized`	250
`nvidia/Nemotron-SFT-OpenCode-v1`	250
`synthetic_complete_code`	80
Total	1080

Approximate text size:

text

examples: 1080
characters: 3,985,263
average characters/example: 3,690.1

The trainer log reported approximately:

num_tokens: 3.899e+06

at the final training step.

Dataset Filtering

The dataset preparation script filtered out examples containing placeholder or incomplete-answer patterns, including:

text

TODO
insert your code here
your code here
left as an exercise
you can continue
and so on
etc.
...
implement the rest
fill in
placeholder

Synthetic examples were added to emphasize complete runnable code and avoidance of placeholder-only answers.

Training Procedure

Training used QLoRA supervised fine-tuning.

Base model path during training:

/workspace/train/qwen35_diligent_lora/models/Qwen3.5-9B-Base

Public base model:

Qwen/Qwen3.5-9B-Base

LoRA Parameters

From adapter_config.json:

text

peft_type: LORA
task_type: CAUSAL_LM
r: 16
lora_alpha: 32
lora_dropout: 0.05
bias: none
use_dora: false
use_rslora: false

Target modules:

text

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

Quantization

Training used 4-bit QLoRA:

text

load_in_4bit: true
bnb_4bit_quant_type: nf4
bnb_4bit_compute_dtype: bfloat16
bnb_4bit_use_double_quant: true

SFT Hyperparameters

text

max_steps: 600
save_steps: 100
max_length / sequence length: 4096
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
effective batch size: 8
learning_rate: 1.5e-4
warmup_ratio: 0.03
lr_scheduler_type: cosine
logging_steps: 5
save_total_limit: 6
bf16: true
fp16: false
gradient_checkpointing: true
optimizer: paged_adamw_8bit
seed: 42
packing: false

Runtime

Final trainer log:

text

train_runtime: 6914 seconds
train_samples_per_second: 0.694
train_steps_per_second: 0.087
epoch: 4.444

Loss Curve

The final training loss was:

train_loss: 0.4767

Selected logged points from the final stage:

Step	Epoch	Loss	Mean token accuracy	Learning rate
540	4.000	0.3872	0.8982	4.029e-06
545	4.037	0.2483	0.9253	3.401e-06
550	4.074	0.3037	0.9060	2.824e-06
560	4.148	0.3212	0.9060	1.829e-06
570	4.222	0.2903	0.9146	1.048e-06
580	4.296	0.3849	0.9048	4.813e-07
590	4.370	0.2959	0.9217	1.322e-07
600	4.444	0.3393	0.9026	1.093e-09

Interpretation: the loss indicates that the adapter fit the SFT mixture, but training loss alone does not prove downstream quality. External evaluation is still required.

Evaluation

No formal benchmark evaluation is included in this release yet.

Recommended evaluation before making quality claims:

Code

HumanEval
MBPP
LiveCodeBench
Repo-level patch tests
Internal unit-test based coding tasks

Translation and Localization

Human review on game dialogue samples
Terminology consistency tests
Format preservation tests
Variable/tag preservation tests
Side-by-side comparison with the base model

General Regression

MMLU-style knowledge checks
GSM/math samples
Summarization and instruction-following checks
Refusal/safety behavior checks if deployed publicly

Until these tests are added, the correct claim is:

This adapter was trained to bias Qwen3.5-9B-Base toward more complete technical and localization-style outputs, but benchmarked improvements over the base model have not yet been established.

Reproducibility

The original training package included:

text

data/sft_mix.jsonl
scripts/prepare_sft_mix.py
scripts/train_qlora.py
scripts/start_train_qlora.sh
logs/train_qwen35_diligent_lora_*.log
outputs/qwen35_diligent_lora_v1/checkpoint-100
outputs/qwen35_diligent_lora_v1/checkpoint-200
outputs/qwen35_diligent_lora_v1/checkpoint-300
outputs/qwen35_diligent_lora_v1/checkpoint-400
outputs/qwen35_diligent_lora_v1/checkpoint-500
outputs/qwen35_diligent_lora_v1/checkpoint-600
outputs/qwen35_diligent_lora_v1/final_adapter

Minimal reproduction command:

bash

source /venv/main/bin/activate
CUDA_VISIBLE_DEVICES=0 python scripts/train_qlora.py \
  --model /path/to/Qwen3.5-9B-Base \
  --dataset data/sft_mix.jsonl \
  --output outputs/qwen35_diligent_lora_v1 \
  --max-steps 600 \
  --save-steps 100 \
  --seq-len 4096 \
  --lr 1.5e-4

Loading With Transformers + PEFT

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "Qwen/Qwen3.5-9B-Base"
adapter_path = "taylonmcfly/Qwen3.5-9B-Existence-Code"

tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_path)
model.eval()

messages = [
    {
        "role": "system",
        "content": "You are Qwen Existence Code, a precise technical assistant.",
    },
    {
        "role": "user",
        "content": "Write a complete Python script that scans a folder and prints file sizes.",
    },
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.4,
        top_p=0.9,
        repetition_penalty=1.1,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Recommended Prompting

text

You are Qwen Existence Code, a precise technical assistant.
Follow the user's task completely.
When writing code, provide complete runnable files or patches unless the user explicitly asks for a sketch.
Do not replace required logic with placeholders such as TODO, "insert code here", or "continue yourself".
Use the provided RAG context as the source of truth when it is relevant.
For translation/localization, preserve meaning, tone, character voice, formatting, variables, markup, and speaker intent.

Recommended Generation Settings

Technical work:

text

temperature: 0.2-0.5
top_p: 0.85-0.95
repetition_penalty: 1.05-1.15
max_new_tokens: high enough for complete output

Translation/localization:

text

temperature: 0.3-0.7
top_p: 0.9
repetition_penalty: 1.05

Limitations

This is a LoRA adapter, not a full standalone model.
It depends on Qwen/Qwen3.5-9B-Base.
It can hallucinate or make mistakes.
Training loss does not prove real-world superiority.
Formal code, translation, and regression benchmarks are not included yet.
Human review is recommended for production code and sensitive translations.
It may require RAG context for project-specific facts.

Responsible Use

This model is a tool. Users are responsible for their inputs, outputs, and deployment choices.

Do not use this model for illegal activity, unauthorized access, fraud, targeted harassment, or other harmful purposes. For legal, medical, financial, security-critical, or production-sensitive work, outputs should be reviewed by a qualified human.

Citation

Qwen Existence Code, PEFT LoRA adapter for Qwen/Qwen3.5-9B-Base.