Loss is applied only to the final structured answer. The training data includes Chain-of-Thought reasoning, but those reasoning tokens are excluded from loss computation so the adapter learns to produce correct output regardless of whether CoT is generated at inference time.

Training Configuration

Parameter	Value
Max Sequence Length	512
Epochs	2
Learning Rate	1e-05
LoRA r	32
LoRA alpha	64
Quantization	4-bit

Intended Use

Recommended:

Generating structured data (JSON, YAML, XML, TOML, CSV) from natural language instructions
Format conversion between structured formats
Data extraction tasks where output must follow a strict schema

Not recommended:

General-purpose chat or creative writing
Tasks requiring outputs longer than 512 tokens
Safety-critical or regulated applications without further validation

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-4B-Instruct-2507"
adapter_model = "hirosan6595/lora_structeval_t_qwen3_4b-55"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(model, adapter_model)

# --- Generate Example ---
prompt = "Convert the following to JSON: name is Alice, age is 30, city is Tokyo."

messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Tip: For best results, give a clear instruction specifying the desired output format (e.g. "Output valid JSON only.").

Dataset

Training dataset: https://huggingface.co/datasets/u-10bei/structured_data_with_cot_dataset_512_v2

Dataset License: MIT License

Base Model Terms

This adapter inherits all usage restrictions of the base model. Please review the base model license before deployment: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Limitations

May still produce invalid structured output in long generations (>512 tokens)
Not optimized for multilingual generation
Performance depends on prompt format and specificity

Citation

bibtex

@misc{lora_structeval_t_qwen3_4b-55,
  author = {hirosan6595},
  title  = {lora_structeval_t_qwen3_4b-55},
  year   = 2026,
  publisher = {Hugging Face},
  url    = {https://huggingface.co/hirosan6595/lora_structeval_t_qwen3_4b-55},
}

License

Repository License: apache-2.0

This license applies to adapter weights only.