Views
No views yet
Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-sharegpt-sw512 |
| Checkpoint | epoch_8_step_415000 |
| Epoch | 8 |
| Global step | 415000 |
| Files | config.json, model.safetensors, training_state.pt |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Framework revision | 9fbbde8ab5d6ee69fb0af3701330027b8beca37a |
| Training data | sharegpt_train_clean.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Epochs configured | 10 |
| Total scheduled steps | 463620 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 7 |
| Draft accumulation steps | 1 |
| Draft sliding window | 512 |
| Save / eval interval | 5000 / 5000 |
| Seed | 0 |
| TP / DP size | 1 / 2 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 16 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 1 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | 512 |
| Max window layers | 28 |
training_state.pt is included for provenance and training-state inspection.