SmolLM2-360M · One-Paragraph Accident Reporter (LoRA)

Task

Training

• Data: ~4500 rows (English), each with 5W1H input and single-line target paragraph.
• Hyperparams: 30 epochs, LR 2e-4 (cosine), warmup 5%, weight decay 5%, eff batch ~64, seq len 1024, optim paged_adamw_8bit, metric: eval_loss
• Hardware: T4 16GB, QLoRA (nf4, double quant).
• Methods: SFTTrainer with early stop (patience=2, threshold=1e-3)
• results: stopped at 8 epochs with best eval loss: 0.6953 at step 426 (perplexity ~ 2.00). Final train loss: 0.5922 at step 560

Inference prompt (recommended)

• Generate only one report given the informations about the event
• Generate the report as text in one paragraph
• It is important to focus on accuracy and coherence when generating the report so that the description content matches the information provided (what, when, where, who, how , why, contingency actions). If an information is not provided in (what, when, where, who, how , why, contingency actions), it must not be part of the generated text description.

License

• Base: Apache-2.0
• LoRA: Apache-2.0

Limitations

• English-focused; short outputs only.

Citation