Views
No views yet
1import pickle
2import numpy as np
3
4# load model
5with open("speckv_mlp16.pkl", "rb") as f:
6 model = pickle.load(f)
7
8# at each speculation step, extract these from draft token distributions:
9draft_entropy = 1.5 # mean entropy across draft tokens
10draft_confidence = 0.72 # mean top-1 confidence
11max_entropy = 2.3 # max entropy in the step
12min_confidence = 0.45 # min confidence in the step
13comp_enc = 0 # 0=fp16, 1=int8, 2=nf4
14
15# pick best gamma
16best_gamma, best_expected = 2, 0
17for gamma in [2, 4, 6, 8]:
18 features = np.array([[draft_entropy, draft_confidence, max_entropy, min_confidence, comp_enc, gamma]])
19 pred_ar = np.clip(model.predict(features)[0], 0, 1)
20 expected_tokens = pred_ar * gamma + 1
21 if expected_tokens > best_expected:
22 best_expected = expected_tokens
23 best_gamma = gamma
24
25print(f"Use gamma={best_gamma} (expected {best_expected:.1f} tokens)")1import numpy as np
2
3weights = np.load("speckv_mlp16_weights.npz")
4W1, b1 = weights["W1"], weights["b1"] # (6, 16), (16,)
5W2, b2 = weights["W2"], weights["b2"] # (16, 1), (1,)
6
7def predict(x):
8 h = np.maximum(0, x @ W1 + b1) # ReLU
9 return float(h @ W2 + b2)| Property | Value |
|---|---|
| Architecture | MLP, 1 hidden layer, 16 units, ReLU |
| Input | 6 features (entropy, confidence, max/min variants, compression, gamma) |
| Output | Acceptance rate prediction (0-1) |
| Training data | 5,112 step-level records |
| Test MSE | 0.090 |
| Test correlation | 0.685 |
| Decision overhead | 0.34ms (4 predictions per decision) |
| Improvement over fixed gamma=4 | 56.0% |
| Statistical significance | p < 0.001 |
speckv_mlp16.pkl - Full scikit-learn model (pickle)speckv_mlp16_weights.npz - Raw numpy weights (W1, b1, W2, b2)config.json - Model configuration and metadatarequirements.txt - Python dependencies1@article{shukla2026speckv,
2 title={SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection},
3 author={Shukla, Shikhar},
4 journal={arXiv preprint},
5 year={2026}
6}