GPT-Neo 125M LoRA Fine-Tuned Review Generator
Model Overview
This model is a review-generation fine-tuned version of EleutherAI/gpt-neo-125M , trained using LoRA (Low-Rank Adaptation) on the Yelp Review Full dataset.
The model generates structured and natural-sounding restaurant and product reviews when prompted in a conversational format.
Fine-tuning was performed using parameter-efficient training, and LoRA weights were merged into the base model for standalone deployment.
Model Details
Model Description
This is a causal language model fine-tuned for review generation.
Expected prompt format:
User: Write a review about a new restaurant.
Assistant:The model generates a realistic review continuation after Assistant:.
Model Type
Causal Language Model (GPT-Neo architecture) fine-tuned using LoRA
Language
English
Base Model
Fine-Tuning Method
Low-Rank Adaptation (LoRA)
LoRA enables efficient fine-tuning by training a very small subset of parameters instead of updating the entire model.
After training:
LoRA weights were merged into the base model
The final model is standalone (no adapter required)
Intended Uses
Direct Use
Synthetic review generation
NLP experimentation
Prompt engineering practice
Educational purposes (fine-tuning & LoRA practice)
Example Usage
1 from transformers import AutoTokenizer , AutoModelForCausalLM
2 import torch
3
4 device = "cuda" if torch . cuda . is_available ( ) else "cpu"
5
6 model = AutoModelForCausalLM . from_pretrained (
7 "rahmaelnabarawy/gptneo125m-finetuned-generate-reviews"
8 ) . to ( device )
9
10 tokenizer = AutoTokenizer . from_pretrained (
11 "rahmaelnabarawy/gptneo125m-finetuned-generate-reviews"
12 )
13
14 prompt = """User: Write a review about a nearby coffee shop.
15 Assistant:"""
16
17 inputs = tokenizer ( prompt , return_tensors = "pt" ) . to ( device )
18
19 outputs = model . generate (
20 ** inputs ,
21 max_new_tokens = 100 ,
22 temperature = 0.8 ,
23 top_p = 0.95 ,
24 do_sample = True
25 )
26
27 print ( tokenizer . decode ( outputs [ 0 ] , skip_special_tokens = True ) )
Training Details
Dataset
Training subset used:
30,000 training samples
2,000 validation samples
Data Format Used for Training
User: Write a review.
Assistant: <review text>The conversational structure was intentionally used to guide generation behavior.
Training Hyperparameters
Parameter Value Base Model GPT-Neo-125M LoRA Rank (r) 16 LoRA Alpha 16 LoRA Dropout 0.05 Epochs 3 Batch Size 8 Gradient Accumulation 2 Learning Rate 2e-4 Block Size 128 Optimizer AdamW Precision FP16 (if GPU available)
Architecture
GPT-Neo Transformer
Layers: 12
Hidden Size: 768
Attention Heads: 12
Parameters: 125M
Objective: Causal Language Modeling
Compute Infrastructure
Hardware:
NVIDIA Tesla T4 GPU (or similar)
Software:
PyTorch
Hugging Face Transformers
PEFT
Hugging Face Datasets
Limitations
Small model size (125M parameters)
May generate repetitive or generic text
Sensitive to prompt format
Inherits biases from Yelp dataset
Not optimized for production-level deployment
Out-of-Scope Use
This model is NOT intended for:
Real-world review manipulation
Deceptive content generation
High-stakes decision systems
