🦙 Mistral-7B Instruct – Fine-Tuned on Databricks Dolly 15k
This model is a fine-tuned version of Mistral-7B-Instruct trained using a subset of the Databricks Dolly 15k instruction dataset.
Training was performed with LoRA using the trl library’s SFTTrainer.
🚀 Model Purpose
The goal of this fine-tuning run is to:
Improve instruction-following abilities
Enhance short-form question-answering
Produce more concise and helpful outputs
Evaluate ROUGE improvements on Dolly-style tasks
This model is not intended to exceed top-tier instruct models but to provide a lightweight, reproducible fine-tune for research and experimentation.
🧠 Base Model
Mistral-7B-Instruct
7B parameters
Optimized for instruction following
Strong general capabilities
License: Apache 2.0
📚 Training Dataset
Dataset:databricks/databricks-dolly-15k Split: 95% train / 5% test Seed:4016
Dolly is a small, mixed-quality instruction dataset containing multiple task types:
