LED Narrative Factual 16k
Warm-up checkpoint for factual chapter summaries, built to ground longer outline models.
- Base model:
allenai/led-base-16384
- Warm start:
paragraph_baseline_models/led_base_baseline/checkpoint-6500
- Dataset:
gemini-chapter-summary.jsonl (~38k chapters, avg ~950 tokens)
- Tokenization: LED tokenizer,
max_source_length=16384, max_target_length=128
- Training: batch 8 (per device) × grad accum 4 (effective 32), lr 2e-5, 3 epochs, linear warmup 1k steps
- Validation metrics (400 chapters, beam=4, length_penalty=0.6):
- ROUGE-1: 0.3523
- ROUGE-2: 0.1045
- ROUGE-L: 0.2490
- METEOR: 0.2372
- Avg generation length: 34.7 words
Use this checkpoint as the factual summarization baseline before training creative long-form outline models.
