You can deploy this model and start earning money today!

skhamzah123/
llama-3.2-1b-prompt-tuning-twitter

Name: llama-3.2-1b-prompt-tuning-twitter
Availability: InStock

safetensors

Views

No views yet

library_name: peft
base_model: meta-llama/Llama-3.2-1B
tags:

prompt-tuning
text-classification
twitter
complaint-detection
peft
datasets:
ought/raft
language:
en
license: llama3.2

Llama 3.2 1B - Prompt Tuned for Tweet Complaint Classification

This model is a prompt-tuned version of Llama 3.2 1B for classifying tweets as "complaint" or "no complaint".
It uses PEFT (Parameter-Efficient Fine-Tuning) with prompt tuning, training only 8,192 parameters (~0.0015% of the base model).

🔍 Model Description

Base Model: meta-llama/Llama-3.2-1B
Method: Prompt Tuning (PEFT)
Task: Binary Text Classification
Trainable Parameters: 8,192 / 1.24B (0.0015%)
Dataset: RAFT – Twitter Complaints subset

🚀 Quick Start

python

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained(
    "skhamzah123/llama-3.2-1b-prompt-tuning-tweet-peft"
).to("cuda")

tokenizer = AutoTokenizer.from_pretrained(
    "skhamzah123/llama-3.2-1b-prompt-tuning-tweet-peft"
)

# Set pad token if missing
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token
    tokenizer.pad_token_id = tokenizer.eos_token_id

# Classify a tweet
tweet = "@customer_service This is unacceptable! Still waiting for help."
inputs = tokenizer(f"Tweet text : {tweet} Label : ", return_tensors="pt").to("cuda")

outputs = model.generate(
    input_ids=inputs["input_ids"],
    max_new_tokens=10,
    pad_token_id=tokenizer.pad_token_id,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📘 Training Details

Training Data

Dataset: ought/raft – Twitter Complaints subset
Classes: complaint, no complaint

Training Hyperparameters

Parameter	Value
Learning Rate	3e-2
Epochs	50
Batch Size	16 (train & eval)
Max Seq Length	64
Optimizer	AdamW
Trainer	TRL `SFTTrainer`

🎛️ Prompt Tuning Configuration

Setting	Value
Init Text	"Classify if the tweet is a complaint or no complaint.
"
Virtual Tokens	13
Init Method	TEXT

📈 Performance

The model efficiently classifies tweets into complaint vs. no complaint with minimal computational overhead due to prompt tuning.

✅ Intended Use

Primary Use:
Automated classification of customer-service-related tweets to detect complaints for routing, analysis, or prioritization.

Limitations:

Trained specifically on Twitter-style text
Works best for customer-service / complaint-oriented contexts
May not generalize outside the domain without further tuning

If you use this model, feel free to ⭐ star the repo and share feedback!

skhamzah123/llama-3.2-1b-prompt-tuning-twitter