You can deploy this model and start earning money today!

littlesprout/
qwen2.5-0.5b-dolly-instruction-tuned

Name: qwen2.5-0.5b-dolly-instruction-tuned
Availability: InStock

transformers datasets peft trl accelerate bitsandbytes>=0.46.1

databricks/databricks-dolly-15k

Qwen/Qwen2.5-0.5B

finetune

Views

No views yet

Model Description

This model is a LoRA adapter fine-tuned from Qwen/Qwen2.5-0.5B to improve instruction-following capabilities. The goal of this project was to see whether fine-tuning a small language model on a subset of instruction-following data could improve its performance on concrete tasks. The model was trained using the databricks/databricks-dolly-15k dataset. Instead of using the entire dataset, in order to train the model for tutor-like behaviour that uses the existing knowledge, classification, creative writing and brainstorming categories were discarded and only the following categories were selected: closed_qa open_qa general_qa information_extraction summarization After filtering, 10400 examples remained. The data was split into 10088 training examples and 312 test examples using a 0.03 split with a random seed of 42 for reproducibility.

Uses

This model is intended as experimentation with small instruction-following language models. The model is particularly suitable for educational on both, basic factual academic question-answering tasks(up to aproximately high school level) and experimental use, including studying the effects of parameter-efficient fine-tuning on a language model where its small size becomes an advantage in this case.

Model Limitations

The model is based on the relatively small Qwen2.5-0.5B base model and may produce less capable responses than larger language models. Even though the model was trained on 10k(after filtering) dataset, it was trained for only one epoch, limiting the improvements to a moderate level. This repository contains LoRA adapter weights and requires the original Qwen/Qwen2.5-0.5B base model for inference.

How to Get Started with the Model

Installation:

pip install transformers peft

Use the code below to get started with the model.

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-0.5B"
adapter_id = "YOUR_USERNAME/YOUR_MODEL_NAME"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id
)

model = PeftModel.from_pretrained(
    base_model,
    adapter_id
)

prompt = "List five titles by Mary Shelley."

messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id
)

generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]

print(
    tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    )
)

Evaluation

The model was evaluated on the same test set before and after fine-tuning using ROUGE metrics. ROUGE1 Before: 0.2010 After: 0.2452 Change: +0.0442 ROUGE2 Before: 0.0942 After: 0.1215 Change: +0.0273 ROUGEL Before: 0.1664 After: 0.2036 Change: +0.0372 Fine-tuning resulted in an improvement across the three evaluated ROUGE metrics. ROUGE-1 increased by approximately 22.0%, ROUGE-2 by approximately 29.0%, and ROUGE-L by approximately 22.4%. Interestingly, the ROUGE-2 has improved considerably more than 1, suggesting that the fine-tuned model is producing more matching two-word sequences than matching single words.

littlesprout/qwen2.5-0.5b-dolly-instruction-tuned