TinyLlama-R1 is a lightweight transformer model designed to handle instruction-following and reasoning tasks, particularly in STEM domains. This model was trained using the Magpie Reasoning V2 250K-CoT dataset, with a goal to improve reasoning through high-quality instruction-response pairs. However, based on early tests, TinyLlama-R1 shows reduced responsiveness to system-level instructions, likely due to the absence of system messages in the dataset.
Model Name: Josephgflowers/Tinyllama-STEM-Cinder-Agent-v1
Key Features
Dataset Focus: Built on the Magpie Reasoning V2 250K-CoT dataset, enhancing problem-solving in reasoning-heavy tasks.
STEM Application: Tailored for tasks involving scientific, mathematical, and logical reasoning.
Instruction Handling: Initial observations indicate reduced adherence to system instructions, a change from previous versions.
Model Details
Model Type: Transformer-based (TinyLlama architecture)
Parameter Count: 1.1B
Context Length: Updated to 8k
Training Framework: Unsloth
Primary Use Cases:
Inteded for research into COT in small language models
Technical problem-solving
Instruction-following conversations
Training Data
The model was fine-tuned on the Magpie Reasoning V2 250K-CoT dataset. The dataset includes diverse instruction-response pairs, but notably lacks system-level messages, which has impacted the model's ability to consistently follow system directives.
Dataset Characteristics
Sources:
Instructions were generated using models like Meta's Llama 3.1 and 3.3.
Responses were provided by DeepSeek-R1-Distill-Llama-70B.
Structure: Instruction-response pairs with an emphasis on chain-of-thought (CoT) reasoning styles.
Limitations: No system-level instructions were included, affecting instruction prioritization and response formatting in some contexts.
Known Issues & Limitations
System Instructions: The model currently does not respond well to system messages, in contrast to previous versions.
Performance Unverified: This version has not yet been formally tested on benchmarks like GSM-8K.
License: CC BY-NC 4.0 (Non-commercial use only)
This model was trained using datasets under:
Meta Llama 3.1 and 3.3 Community License
CC BY-NC 4.0 (Creative Commons Non-Commercial License)
Acknowledgments
Thanks to the Magpie Reasoning V2 dataset creators and the researchers behind models like Deepseek-R1 and Meta Llama.
@article{xu2024magpie,
title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing},
author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin},
year={2024},
eprint={2406.08464},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
