This model is a fine-tuned version of meta-llama/Llama-2-7b-hf using PEFT (LoRA) on a custom 20-example K-Pop Q&A dataset.
✨ Overview
Task: Text generation (K-Pop Question Answering)
Dataset: Manually written 20 K-Pop themed examples
Training Setup: LoRA-based fine-tuning on Google Colab Free Tier (15 GB GPU)
Use Case: Lightweight model for educational/demo use related to K-Pop fan Q&A
📚 Dataset Description
Each example is structured as:
Example:
text
1Question: Who is the leader of BTS?
2Context: BTS is a popular South Korean boy band formed in 2013.
3Answer: RM is the leader of BTS.
🧪 Training Details
Parameter
Value
Base model
LLaMA2 7B HF
Finetuning method
PEFT (LoRA)
Epochs
1
Batch Size
1
Max Length
512 tokens
Optimizer
AdamW
Learning Rate
2e-4
GPU Used
Free Google Colab
Layers Updated
Only 1 (rest frozen)
Quantization
4-bit (bitsandbytes)
🧾 How to Use
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model = AutoModelForCausalLM.from_pretrained("Tammy7777777/kpop-llama2-finetuned", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Tammy7777777/kpop-llama2-finetuned")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
prompt = "Question: Who is the leader of BTS?\nContext: BTS is a South Korean boy band formed in 2013.\nAnswer:"
print(pipe(prompt, max_new_tokens=64)[0]['generated_text'])
📈 Evaluation
Base model output: Often generic or unrelated
Fine-tuned model: More aligned, provides specific K-Pop answers
Method: Manual comparison of predictions before & after fine-tuning
📦 Model Architecture
Based on meta-llama/Llama-2-7b-hf
Only one transformer layer fine-tuned using LoRA
Efficient for few-shot adaptation
⚠️ Limitations
Trained on a very small (20-example) dataset
May hallucinate or overfit on known examples
Not suitable for production use or factually sensitive topics
👤Author
Tamanna Sheikh (@Tammy7777777)
📝 License
This model is released under the MIT license for research and educational use.
