This model is a fine-tuned version of Qwen3-0.6B specifically optimized for binary sentiment classification (positive/negative) on movie reviews. It uses LoRA adaptation to efficiently fine-tune the model while maintaining a small memory footprint.

Developed by: [Yiwen]
Model type: Causal Language Model with LoRA adaptation for sequence classification
Language(s) (NLP): English
License: Same as base model (Apache 2.0)
Finetuned from model: Qwen/Qwen3-0.6B

Model Sources

Repository: https://huggingface.co/yiwenX/Qwen3-0.6B-imdb
Base Model: https://huggingface.co/Qwen/Qwen3-0.6B

Uses

Direct Use

This model can be used directly for sentiment analysis of English movie reviews or similar text. It classifies text into two categories:

Label 0: Negative sentiment
Label 1: Positive sentiment

Downstream Use

The model can be further fine-tuned for:

Other binary text classification tasks
Domain-specific sentiment analysis (product reviews, social media, etc.)
Multi-class sentiment analysis with additional training

Out-of-Scope Use

The model is trained on English movie reviews and may not perform well on:
- Non-English text
- Highly technical or domain-specific content
- Text significantly different from movie reviews
Should not be used for critical decision-making without human oversight

Bias, Risks, and Limitations

Dataset Bias: IMDB dataset may contain biases related to movie genres, time periods, and reviewer demographics
Language Limitation: English-only model
Domain Specificity: Optimized for movie reviews, performance may degrade on other text types
Binary Classification: Only distinguishes positive/negative, cannot detect neutral sentiment

How to Get Started with the Model

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

# 直接从HuggingFace加载你的模型
model_name = "yiwenX/Qwen3-0.6B-imdb"
model = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# 如果tokenizer没有pad_token，设置一下
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# 推理示例
def predict_sentiment(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512, padding=True)
    
    # 移动到正确的设备
    inputs = {k: v.to(model.device) for k, v in inputs.items()}
    
    with torch.no_grad():
        outputs = model(**inputs)
        prediction = torch.argmax(outputs.logits, dim=-1)
    
    return "Positive" if prediction.item() == 1 else "Negative"

# 测试示例
test_texts = [
    "This movie was absolutely fantastic! Great acting and storyline.",
    "Terrible movie, waste of time. Bad acting and boring plot.",
    "One of the best films I've ever seen. Highly recommend!",
    "I fell asleep halfway through. Very disappointing."
]

for text in test_texts:
    sentiment = predict_sentiment(text)
    print(f"Text: {text[:50]}...")
    print(f"Sentiment: {sentiment}\n")

Training Details

Training Data

Dataset: IMDB Movie Review Dataset
Training samples: 25,000
Validation samples: 25,000
Data source: https://huggingface.co/datasets/imdb

Training Procedure

Preprocessing

Tokenization using Qwen tokenizer
Maximum sequence length: 512 tokens
Padding to max length
Truncation of longer sequences

Training Hyperparameters

Training regime: fp16 mixed precision
LoRA rank (r): 16
LoRA alpha: 32
LoRA dropout: 0.1
Target modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
Batch size: 8 per device
Gradient accumulation steps: 4
Learning rate: 2e-4
Number of epochs: 3
Warmup steps: 100
Weight decay: 0.01
Optimizer: AdamW

Speeds, Sizes, Times

Training time: ~50 minutes on single GPU
Model size: ~10MB (LoRA weights only)
Base model size: ~1.2GB
Training speed: ~0.72 iterations/second

Evaluation

Testing Data, Factors & Metrics

Testing Data

IMDB test set (25,000 samples)
Balanced dataset (50% positive, 50% negative)

Metrics

Accuracy: Primary metric for balanced binary classification
F1 Score: Harmonic mean of precision and recall
Precision/Recall: For detailed performance analysis

Results

Metric	Value
Test Accuracy	~92-94% (expected)
F1 Score	~0.92-0.94
Training Loss (final)	~0.1-0.2

Note: Exact values depend on random seed and training conditions

Environmental Impact

Hardware Type: NVIDIA GPU (13GB VRAM)
Hours used: ~1 hour
Cloud Provider: [Local]
Carbon Efficiency: Minimal due to LoRA efficiency

Technical Specifications

Model Architecture and Objective

Base Architecture: Qwen3 (Transformer-based)
Parameters: 606M total, 10M trainable (1.66%)
Objective: Cross-entropy loss for binary classification
Adaptation Method: LoRA (Low-Rank Adaptation)

Compute Infrastructure

Hardware

Single GPU with 13GB+ VRAM
CUDA-capable device

Software

PyTorch 2.0+
Transformers 4.30+
PEFT 0.15.2
CUDA 11.7+

Citation

bibtex

@misc{qwen3-imdb-lora,
  author = {yiwenX},
  title = {Qwen3-0.6B-IMDB: Fine-tuned Sentiment Analysis Model},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/yiwenX/Qwen3-0.6B-imdb}
}

Model Card Contact

yiwenlemo@gmail.com

Framework versions

PEFT 0.15.2
Transformers 4.30+
PyTorch 2.0+