u-10bei系: 「Output:」以降だけを抽出
- Problem: u-10bei datasets contain "Approach: → Output:" structure
- Solution: Extract only content after "Output:" marker
- Effect: Removes "explanation before output" tendency
daichira系: 2倍ブースト（「Return ONLY」パターン）
- Problem: Model needs more "Return ONLY YAML/XML" examples
- Solution: Boost daichira datasets by 2x
- Effect: Strengthens "direct output without explanation" pattern
YAML/XML: 2倍ブースト
- Problem: YAML/XML are the weakest formats (94.3%, 85.0%)
- Solution: Boost YAML/XML samples by 2x
- Effect: More training data for weak formats

Data Cleaning Pipeline:

CoT tags removal: <thinking>...</thinking> completely removed
Code fence removal: yaml, json, xml, toml, ````csv removed
Leading phrase removal: "Here's the output:", "Sure!", etc. removed
🔥 Output extraction: For u-10bei datasets, extract only content after "Output:" marker
Format validation: JSON/YAML/XML/TOML/CSV parsing validation
Deduplication: Exact duplicates removed

Format Distribution (Estimated):

YAML: ~12,000-15,000 (40-50%) 🔥 2x boost
XML: ~6,000-8,000 (25-30%) 🔥 2x boost
JSON: ~4,000-5,000 (15-20%)
TOML: ~2,500-3,000 (10%)
CSV: ~2,500-3,000 (10%)

Total: ~25,000-30,000 samples

Source Datasets:

u-10bei系（「Output:」抽出適用）:

u-10bei/structured_data_with_cot_dataset_512_v2
u-10bei/structured_data_with_cot_dataset_512_v4
u-10bei/structured_data_with_cot_dataset_512_v5
u-10bei/structured_data_with_cot_dataset_512
u-10bei/structured_data_with_cot_dataset_v2
u-10bei/structured_data_with_cot_dataset

daichira系（2倍ブースト）:

daichira/structured-3k-mix-sft 🔥 2x
daichira/structured-5k-mix-sft 🔥 2x
daichira/structured-hard-sft-4k 🔥 2x

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "yuk1chan/qwen3-4b-structeval-yamlxml-boost-v2-lr6e-6"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

# Inference
prompt = "Generate YAML code for..."
# ... your inference code

Training Results

- Training Loss: ~1.25-1.30
- Validation Loss: ~1.50-1.55
- Training Time: ~8-9 hours
- Expected Score: 0.805-0.810 (YAML/XML improvement from 0.80195 baseline)

Strategy: YAML/XML Clean Expansion (Revised)

Key Insights from Expert AI Analysis:

The main causes of YAML/XML failures are:
1. Code fences (yaml, xml) appearing in output
2. Leading phrases ("Here's the output:", "Sure!") before output
3. u-10bei datasets' "Approach: → Output:" structure leaking into outputs

Solutions Implemented:
┌─────────────────────────────────────┬──────────────────────────────┬───────────────────────────────────┐
│               Problem               │           Solution           │              Effect               │
├─────────────────────────────────────┼──────────────────────────────┼───────────────────────────────────┤
│ u-10bei "Output:" structure         │ Extract only after "Output:" │ Removes explanation tendency      │
├─────────────────────────────────────┼──────────────────────────────┼───────────────────────────────────┤
│ Insufficient "Return ONLY" examples │ Boost daichira datasets 2x   │ Strengthens direct output pattern │
├─────────────────────────────────────┼──────────────────────────────┼───────────────────────────────────┤
│ YAML/XML too weak                   │ Boost YAML/XML 2x            │ More training data                │
└─────────────────────────────────────┴──────────────────────────────┴───────────────────────────────────┘
Expected Improvements:
┌────────┬──────────┬────────┬──────────────────────────┐
│ Format │ Baseline │ Target │         Strategy         │
├────────┼──────────┼────────┼──────────────────────────┤
│ YAML   │ 94.3%    │ 96-97% │ Clean u-10bei + 2x boost │
├────────┼──────────┼────────┼──────────────────────────┤
│ XML    │ 85.0%    │ 88-90% │ Clean u-10bei + 2x boost │
├────────┼──────────┼────────┼──────────────────────────┤
│ TOML   │ 100.0%   │ 100%   │ Maintain                 │
├────────┼──────────┼────────┼──────────────────────────┤
│ JSON   │ 96.0%    │ 96%+   │ Maintain                 │
├────────┼──────────┼────────┼──────────────────────────┤
│ CSV    │ 100.0%   │ 100%   │ Maintain                 │
└────────┴──────────┴────────┴──────────────────────────┘
License

Apache 2.0

---
Trained on Cleaned StructEval dataset (YAML/XML Boosted - Revised)
Learning Rate: 6e-6 (conservative setting)
Strategy: u-10bei Output extraction + daichira 2x + YAML/XML 2x
Based on expert AI analysis