Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
QwQ-14B-Math-v0.2-GGUF – AI Model by QuantFactory | AlphaNeural AI
You can deploy this model and start earning money today!
QuantFactory
/
QwQ-14B-Math-v0.2-GGUF
like
0
conversational
qingy2024/QwQ-LongCoT-Verified-130K
en
endpoints_compatible
gguf
template
apache-2.0
qwen2
us
sft
text-generation-inference
transformers
trl
unsloth
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
QuantFactory Banner
QuantFactory/QwQ-14B-Math-v0.2-GGUF
This is quantized version of
qingy2024/QwQ-14B-Math-v0.2
created using llama.cpp
Original Model Card
Uploaded model
Developed by:
qingy2024
License:
apache-2.0
Finetuned from model :
unsloth/qwen2.5-14b-bnb-4bit
This model is a fine-tuned version of
Qwen 2.5-14B
, trained on QwQ 32B Preview's responses to questions from the
NuminaMathCoT
dataset.
Note:
This model uses the standard ChatML template.
At 500 steps, the loss was plateauing so I decided to stop training to prevent excessive overfitting.
Training Details
Base Model
: Qwen 2.5-14B
Fine-Tuning Dataset
: Verified subset of
NuminaMathCoT
using Qwen 2.5 3B Instruct as a judge. (the
sharegpt-verified-cleaned
subset from my dataset).
QLoRA Configuration
:
Rank
: 32
Rank Stabilization
: Enabled
Optimization Settings
:
Batch Size: 8
Gradient Accumulation Steps: 2 (Effective Batch Size: 16)
Warm-Up Steps: 5
Weight Decay: 0.01
Training Steps
: 500 steps
Hardware Information
: A100-80GB
