Item	Value
Base model	Qwen/Qwen3.5-0.8B
Task	Reddit post → short TL;DR-style summary
Training type	Causal LM SFT (completion-only cross-entropy)
Selected checkpoint	End of epoch 1 (global step 1292)
Validation `eval_loss` (epoch 1)	~1.868

Training data & filtering

Dataset: CarperAI/openai_summarize_tldr (train / valid)
Filters (aligned with AL_Uncertainty/src/data_utils.py):
- Prompt length 200–1000 characters (after rstrip)
- Exclude posts mentioning r/offmychest or r/tifu
Supervision: human label (reference TL;DR) as the assistant turn in a chat template.

After filtering, the training split size is ~20.6k examples.

Training procedure

Training was run with Hugging Face TRL

Key hyperparameters (sft/config/default.yaml):

Hyperparameter	Value
Epochs (planned)	3
Checkpoint released	Epoch 1 only
Per-device train batch size	2
Gradient accumulation	8 → effective batch 16
Learning rate	2e-5
Warmup ratio	0.05
Max sequence length	2048
Precision	bf16

How to use

Load model

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Maximus200005/qwen35-0.8b-tldr-sft-ep1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    use_safetensors=True,
)