Views
No views yet
Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-stage2-adapter-w4 |
| Checkpoint | epoch_7_step_187120 |
| Epoch | 7 |
| Global step | 187120 |
| Files | config.json, model.safetensors, training_state.pt |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training with stage2 adapter |
| Dataset | ShareGPT-format SpecForge training set |
| Learning rate | 1e-4 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 8 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 7 |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 1 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Stage2 adapter | enabled |
| Stage2 adapter bottleneck | 256 |
| Stage2 max depth | 7 |
| Stage2 depth power | 2.0 |
training_state.pt is included for provenance and training-state inspection.