Views
No views yet
Qwen/Qwen3-1.7B.
It is a draft model for speculative decoding, not a standalone target language model.| Field | Value |
|---|---|
| Source run | qwen3-1.7b-eagle3-k2-sw256-sharegpt |
| Checkpoint | epoch_2_step_65000 |
| Epoch | 2 |
| Global step | 65000 |
| Files | config.json, model.safetensors, training_state.pt (when present) |
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Method | SpecForge EAGLE3 online training |
| Training data | sharegpt_train.jsonl |
| Learning rate | 0.0001 |
| Batch size | 1 |
| Target batch size | 1 |
| Epochs configured | 10 |
| Max length | 2048 |
| Warmup ratio | 0.015 |
| Max grad norm | 0.5 |
| TTT length | 5 |
| Draft layers | 2 |
| Draft sliding window | 256 |
| Save interval | 5000 |
| Eval interval | 5000 |
| Seed | 0 |
| TP / DP size | 1 / 4 |
| Attention backend | sdpa |
| Target model backend | sglang |
| SGLang attention backend | flashinfer |
| Dataset build workers | 64 |
| Field | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 |
| dtype | bfloat16 |
| Hidden size | 2048 |
| Intermediate size | 6144 |
| Draft layers | 2 |
| Attention heads | 16 |
| KV heads | 8 |
| Draft vocab size | 32000 |
| Vocab size | 151936 |
| Max position embeddings | 40960 |
| Sliding window | 256 |
| Max window layers | 2 |
| Future hidden | Not recorded |
training_state.pt is included when available for provenance and training-state inspection.