Views
No views yet
zBotta/traffic-accidents-reports-5k.Performance note: In quick spot-checks, this distilled model did not outperform the baselinezBotta/smollm2-accident-reporter-360m-800on Cross-Encoding similarity under the same settings.
Instruction:
You are a reporting agent.
You task is to create a report when provided the what, when, why, who, how and where questions about the events.
You are also given information about the contingency actions regarding the event.
Guidelines:
Generate only one report given the informations about the event
Generate the report as text in one paragraph
It is important to focus on accuracy and coherence when generating the report so that the description content matches the information provided (what, when, where, who, how , why, contingency actions).
If an information is not provided in (what, when, where, who, how , why, contingency actions), it must not be part of the generated text description.
Input:
What: ...
When: ...
Where: ...
Who: ...
How: ...
Why: ...
ContingencyActions: ...
Response:1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_id = "zBotta/SmolLM2-360M-AccidentReports-distilled-kd1.7B"
4tok = AutoTokenizer.from_pretrained(model_id)
5model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto") # merged FP16 weights
6
7prompt = """### Instruction:
8You are a reporting agent...
9[full instruction text as above]
10
11### Input:
12What: rear-end collision between car A and van B at a traffic light
13When: 2025-05-17 08:25 (occurrence)
14Where: Main St & 3rd Ave, downtown
15Who: Driver A (car), Driver B (van); police on scene
16How: A failed to stop in time at red
17Why: suspected distraction; investigation pending
18ContingencyActions: police report filed; medical check for minor neck pain; vehicles towed
19
20### Response:
21"""
22
23inputs = tok(prompt, return_tensors="pt").to(model.device)
24gen = model.generate(
25 **inputs,
26 do_sample=False, # deterministic decoding
27 max_new_tokens=256,
28 eos_token_id=tok.eos_token_id,
29 pad_token_id=tok.pad_token_id,
30 no_repeat_ngram_size=4,
31 repetition_penalty=1.05,
32 renormalize_logits=True,
33)
34print(tok.decode(gen[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())1SFTConfig(
2 output_dir=OUT_DIR,
3 num_train_epochs=20,
4 per_device_train_batch_size=4,
5 gradient_accumulation_steps=16, # effective batch ≈ 64
6 learning_rate=2e-5,
7 lr_scheduler_type="cosine",
8 warmup_ratio=0.1,
9 weight_decay=0.05,
10 label_smoothing_factor=0.05,
11 max_grad_norm=0.5,
12 logging_steps=50,
13 eval_strategy="epoch",
14 save_strategy="epoch",
15 save_total_limit=2,
16 load_best_model_at_end=True,
17 metric_for_best_model="eval_loss",
18 greater_is_better=False,
19 fp16=False, bf16=False, # training precision handled by bnb/device
20 optim="adamw_bnb_8bit",
21 packing=False,
22 max_length=1024,
23 gradient_checkpointing=True,
24 gradient_checkpointing_kwargs={"use_reentrant": False},
25 remove_unused_columns=False,
26 dataloader_num_workers=4,
27 dataloader_pin_memory=True,
28 report_to="none",
29 seed=42,
30)
311@misc{accident_reporter_360m_distilled_kd1.7B,
2 title = {Accident Reporting distilled kd model (One-Paragraph)},
3 author = {zBotta, SamdGuizani},
4 year = {2025}
5}