You can deploy this model and start earning money today!

naksyu/
yui-math-python-qwen35-4b-v0.5d-fft

Name: yui-math-python-qwen35-4b-v0.5d-fft
Availability: InStock

Views

No views yet

yui-math-python-qwen35-4b-v0.5d-fft

Full-parameter SFT of Qwen/Qwen3.5-4B for Korean/English math reasoning, Python-assisted calculation habits, and compact practical answers.

This checkpoint is not a general-purpose assistant release. It is a narrow experimental model intended to test whether a 4B Qwen3.5-style model can learn a stable routine of framing math problems, using executable Python for nontrivial arithmetic, checking edge cases, and ending with a clear answer.

Base Model

Base checkpoint: Qwen/Qwen3.5-4B
Architecture in this export: Qwen3_5ForCausalLM
Native context in config: 262,144 tokens
Training precision: bf16

Intended Use

Use this model for:

Korean and English math problem solving
Python-style calculation planning
Compact verification-oriented explanations
Local experimentation with Qwen3.5 4B SFT behavior

Do not use it as-is for high-stakes math, legal, medical, financial, safety, or production decision making. The dataset includes synthetic and translated reasoning data, and outputs should be checked independently.

Training Summary

The final training file was:

qwen35_sft_v0_5d_system_mix_with_all_data_cutoff2048.jsonl

Local preprocessing stats:

Item	Count
Pre-cutoff rows	51,116
Final retained rows	50,429
Dropped by 2048-token cutoff	687
Min tokens	24
Average tokens	165.18
P95 tokens	546
Max tokens after cutoff	2,048

Training settings:

Setting	Value
Method	full-parameter SFT
Epochs	1
Max sequence length	2,048
Per-device batch size	2
Gradient accumulation	4
Learning rate	1e-5
Scheduler	cosine
Warmup ratio	0.03
Optimizer	adamw_torch
Seed	42
Packing	false
Gradient checkpointing	true

Framework versions recorded by the trainer:

TRL: 1.7.0
Transformers: 5.12.1
PyTorch: 2.8.0+cu128
Datasets: 5.0.0
Tokenizers: 0.22.2

Data Sources

The final training mix combines local Yui/Lime math-style data with two public reasoning-data inputs. Counts below are after deduplication and the 2048-token cutoff used for this run.

Bucket	Source file in build	Retained rows	Notes
DeepSeek-derived Korean sample	`translated_1000.jsonl`	462	Korean translation sample derived from DeepSeek-V4-Flash traces
Claude reasoning trace dataset	`reasoning_merged_ko_filtered_v2.parquet`	11,973	Korean Claude Opus reasoning dataset converted to Qwen chat format
Local/Yui/Lime/generated data	multiple local JSONL files	37,994	Project-specific math, Python, alignment, and short-answer data
Total		50,429

DeepSeek-derived data

Imported source:

drlee1/deepseek-v4-distill-ko-1k

The local dataset card identifies this as a 1,000-row Korean translation sample derived from:

The rows used here had teacher_model=DeepSeek-V4-Flash in their metadata. Of 999 importable rows in the pre-cutoff training file, 462 remained after the 2048-token cutoff.

Claude/trace data

Imported source:

Jongsim/claude-opus-4.6-reasoning-12k-ko-filtered-v2

The local dataset card lists these upstream sources:

Retained rows by upstream source field:

Upstream source	Retained rows
`Roman1111111/claude-opus-4.6-10000x`	9,577
`nohurry/Opus-4.6-Reasoning-3000x-filtered`	2,248
`TeichAI/claude-4.5-opus-high-reasoning-250x`	148
Total	11,973

Important conversion note: the source parquet includes richer message objects, including reasoning metadata in some rows. The local converter for this model kept normalized role and content fields for SFT. Auxiliary message fields such as separate reasoning keys were not intentionally retained as separate training targets.

Prompt Format

Use the tokenizer chat template. The model was trained on Qwen-style chat rendering.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "naksyu/yui-math-python-qwen35-4b-v0.5d-fft"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "user",
        "content": "Solve 3x + 6 = 27. Verify the solution by substitution.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

out = model.generate(inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

GGUF Conversion Note

For GGUF conversion, use a recent llama.cpp build with Qwen3.5 support. This exported config contains mtp_num_hidden_layers=1, but the exported checkpoint used in the local GGUF test did not include separate mtp.* tensors. If your converter emits a GGUF that fails with a missing blk.32.* tensor, reconvert with --no-mtp.

The locally validated Q6_K conversion used the --no-mtp path and loaded in llama-server.

Limitations

This is an experimental SFT checkpoint, not a benchmarked production model.
Source data contains synthetic, translated, and model-generated examples.
The DeepSeek-derived sample contains preserved <think>...</think> style traces in the translated source data.
The Claude trace source was normalized to chat role/content during this build; not every upstream metadata field is represented in the final SFT text.
Some local data is project-specific Yui/Lime behavior data, so the model may prefer compact practical answers over exhaustive proofs.
The model can still make arithmetic or reasoning mistakes. Use Python or an external verifier for important answers.

Source Citation Snippets

bibtex

@misc{qwen35_4b,
  title = {Qwen3.5-4B},
  author = {Qwen Team},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Qwen/Qwen3.5-4B}
}

@misc{deepseek_v4_distill_ko_1k,
  title = {DeepSeek-V4-Distill Korean Translation 1K Sample},
  author = {drlee1},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/drlee1/deepseek-v4-distill-ko-1k}
}

@misc{deepseek_v4_distill_8000x,
  title = {DeepSeek-V4-Distill-8000x},
  author = {Jackrong},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/Jackrong/DeepSeek-V4-Distill-8000x}
}

@misc{glm51_reasoning_1m_cleaned,
  title = {GLM-5.1-Reasoning-1M-Cleaned},
  author = {Jackrong},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/Jackrong/GLM-5.1-Reasoning-1M-Cleaned}
}

@misc{claude_opus_reasoning_12k_ko_filtered_v2,
  title = {Claude Opus Reasoning 12K Korean Filtered v2},
  author = {Jongsim},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/Jongsim/claude-opus-4.6-reasoning-12k-ko-filtered-v2}
}

Training Data

This model was fine-tuned for experimental math/Python reasoning behavior.

The training mix includes:

User-created Lime/Yui math-python SFT data
Public Hugging Face datasets including:
- drlee1/deepseek-v4-distill-ko-1k (MIT)
- Jongsim/claude-opus-4.6-reasoning-12k-ko-filtered-v2 (Apache-2.0)

Some upstream samples are synthetic or translated model outputs. This release is intended as a research/experimental small-model fine-tune, not as a commercial substitute for the upstream model providers.