We introduce SmallThinker-3B-preview, a new model fine-tuned from the Qwen2.5-3b-Instruct model.
Now you can directly deploy SmallThinker On your phones with PowerServe.
Benchmark Performance
Model
AIME24
AMC23
GAOKAO2024_I
GAOKAO2024_II
MMLU_STEM
AMPS_Hard
math_comp
Qwen2.5-3B-Instruct
6.67
45
50
35.8
59.8
-
-
SmallThinker
16.667
57.5
64.2
57.1
68.2
70
46.8
GPT-4o
9.3
-
-
-
64.2
57
50
Limitation: Due to SmallThinker's current limitations in instruction following, for math_comp we adopt a more lenient evaluation method where only correct answers are required, without constraining responses to follow the specified AAAAA format.
SmallThinker is designed for the following use cases:
Edge Deployment: Its small size makes it ideal for deployment on resource-constrained devices.
Draft Model for QwQ-32B-Preview: SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model. From my test, in llama.cpp we can get 70% speedup (from 40 tokens/s to 70 tokens/s).
