merged_exp01
This repository provides a fine-tuned model developed for the LLM Engineering Main Competition.
The model is trained to improve structured output generation for tasks requiring machine-readable formats such as JSON.
Model Overview
Model name
merged_exp01
Base model
unsloth/Qwen3-4B-Instruct-2507
Training approach
Supervised Fine-Tuning (SFT)
Objective
Improve accuracy of structured outputs such as:
JSON
YAML
CSV
XML
The training objective focuses on generating valid structured outputs suitable for automated evaluation pipelines.
Training Data
The model was trained using datasets provided for the competition.
Examples include:
structured_data_with_cot_dataset_512_v2
structured_data_with_cot_dataset_512_v4
structured_data_with_cot_dataset_512_v5
structured_data_with_cot_dataset
structured-3k-mix-sft
structured-5k-mix-sft
structured-hard-sft-4k
These datasets are designed for structured output tasks with reasoning support (CoT).
Note:
Use of these datasets does not guarantee higher scores and participants may combine or customize them.
メインコンペ_使用データまとめ.txt
Training Configuration
Typical training configuration used in this experiment:
Base Model
unsloth/Qwen3-4B-Instruct-2507
Method
QLoRA (4-bit)
Max sequence length
512
Epochs
1
Learning rate
1e-4
LoRA parameters
r = 64
alpha = 128
Usage
Example code for loading the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "i-KenTanaka/merged_exp01"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto"
)
prompt = "Generate structured JSON output for the given task."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Competition Submission
This model is submitted as part of the Main Competition submission requirements.
Submission components:
Inference JSON generated using the official inference code
Public Hugging Face model repository (this repository)
Limitations
The model is optimized for structured output tasks.
Performance may degrade for general conversational tasks.
Structured output validity depends on prompt design.
License and Compliance
Users must comply with:
The license of the base model
The license of the training datasets
Attribution requirements must be respected when using the datasets.
Author
Acknowledgements
This model was developed using:
Hugging Face Transformers
Unsloth
QLoRA training method
Competition datasets provided by the organizers
