You can deploy this model and start earning money today!

tahamajs/
LLaMA-3-2-LoRA-EmotionTune-Full

Name: LLaMA-3-2-LoRA-EmotionTune-Full
Availability: InStock

peft

safetensors

dair-ai/emotion

meta-llama/Llama-3.2-1B

adapter

mit

Views

No views yet

Model Card for LLaMA-3-2-LoRA-EmotionTune

Model Details

Model Description:
LLaMA-3-2-LoRA-EmotionTune is a causal language model fine-tuned using Low-Rank Adaptation (LoRA) on a curated emotion dataset. The dataset consists of user-generated text annotated with emotion labels (sadness, joy, love, anger, fear, or surprise). This fine-tuning enables the model to perform efficient emotion classification while preserving the core strengths of the base LLaMA-3.2-1B-Instruct model.
Developed by:
Taha Majlesi
Funded by (optional):
tahamajs
Shared by (optional):
tahamajs
Model type:
Causal Language Model with LoRA-based fine-tuning for emotion classification
Language(s) (NLP):
English
License:
[Choose a license, e.g., Apache-2.0, MIT]
Finetuned from model (optional):
LLaMA-3.2-1B-Instruct
Model Sources (optional):
Original LLaMA model and publicly available emotion datasets
Repository:
https://huggingface.co/your-username/LLaMA-3-2-LoRA-EmotionTune
Paper (optional):
For LoRA: LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
For LLaMA: [Reference paper details if available]
Demo (optional):
[Link to interactive demo if available]

Uses

Direct Use:
Emotion classification and sentiment analysis on short text inputs.
Downstream Use (optional):
Can be integrated into affective computing systems, chatbots, content moderation pipelines, or any application requiring real-time sentiment detection.
Out-of-Scope Use:
Not recommended for critical decision-making systems (e.g., mental health diagnostics) or for applications in languages other than English without further adaptation.

Bias, Risks, and Limitations

Bias and Risks:
The model may inherit biases present in the training data, potentially misclassifying nuanced emotions or reflecting cultural biases in emotional expression.
Limitations:
- Limited to six predefined emotion categories.
- Performance may degrade for longer texts or in ambiguous contexts.
- The model is fine-tuned on a specific emotion dataset and may not generalize well across all domains.

Recommendations

Users (both direct and downstream) should be aware of the model’s inherent biases and limitations. We recommend additional validation and fine-tuning before deploying this model in sensitive or high-stakes environments.

How to Get Started with the Model

To load and use the model with Hugging Face Transformers and the PEFT library, try the following code snippet:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/LLaMA-3.2-1B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/LLaMA-3.2-1B-Instruct")

# Load fine-tuned LoRA adapter
model = PeftModel.from_pretrained(base_model, "your-username/LLaMA-3-2-LoRA-EmotionTune")

# Example usage:
input_text = "I feel so happy today!"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Demo: An interactive demo for LLaMA-3-2-LoRA-EmotionTune is available on Hugging Face Spaces at https://huggingface.co/spaces/your-username/demo-name.

Training Details

Training Data:
A curated emotion dataset consisting of user-generated text annotated with emotion labels (sadness, joy, love, anger, fear, surprise).
Training Procedure:
Fine-tuning was performed on the LLaMA-3.2-1B-Instruct model using the LoRA method, which adapts selected attention layers using a low-rank approach. This method leverages a small subset of trainable parameters while keeping the majority of the model frozen.
Preprocessing (optional):
Text normalization, tokenization using the Hugging Face tokenizer, and train/validation splitting.
Training Hyperparameters:
- LoRA rank (r): 16
- lora_alpha: 32
- lora_dropout: 0.1
- Learning rate: 2e-5
- Batch size: 32
- Epochs: Early stopping applied around epoch 10 to prevent overfitting.
Speeds, Sizes, Times (optional):
Training conducted on an NVIDIA Tesla T4 GPU for approximately 10–12 hours.

Evaluation

Testing Data:
A held-out subset of the emotion-annotated dataset (e.g., 100 samples).
Factors:
The evaluation focused on the model’s ability to classify emotions within short text outputs.
Metrics:
Accuracy and Micro F1 score.
Results:
The fine-tuned model achieved an accuracy and Micro F1 score of approximately 31% on short-text generation tasks (5–100 token outputs), outperforming the base and instruction-tuned models.

Technical Specifications

Model Architecture and Objective:
Based on LLaMA-3.2-1B-Instruct, the model is fine-tuned using LoRA to specifically classify text into emotion categories.
Compute Infrastructure:
Hugging Face Transformers, PEFT library, and PyTorch.
Hardware:
NVIDIA Tesla T4 or equivalent GPU.
Software:
Python, PyTorch, Hugging Face Transformers, PEFT 0.14.0.

Citation

BibTeX:

bibtex

@inproceedings{hu2021lora,
  title={LoRA: Low-Rank Adaptation of Large Language Models},
  author={Hu, Edward J and others},
  booktitle={Proceedings of ICLR},
  year={2021}
}

APA:

Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of ICLR.

Additional Information

Model Card Authors:
Taha Majlesi
Model Card Contact:
tahamajlesi@ut.ac.ir
Framework Versions:
- PEFT: 0.14.0
- Transformers: [version]
- PyTorch: [version]

tahamajs/LLaMA-3-2-LoRA-EmotionTune-Full