You can deploy this model and start earning money today!

mahi1432/
gpt2-medium-LoRA

Name: gpt2-medium-LoRA
Availability: InStock

peft

safetensors

openai-community/gpt2

adapter

Views

No views yet

Model Card for Story Generation Model (Fine-tuned GPT-2 Medium)

Model Details

Model Description

This model is a fine-tuned version of GPT-2 Medium on the BookCorpus dataset using PEFT (LoRA) for parameter-efficient training. It is intended for story generation tasks, producing creative and coherent narrative text given a prompt.

Developed by: Mahesh Dindur
Institution: KLE Dr. MSSCET, Belagavi
Funded by: N/A (individual project)
Shared by: Mahesh Dindur
Model type: Causal Language Model (CLM)
Language(s) (NLP): English
License: MIT (Open Source)
Finetuned from model: GPT-2 Medium

Model Sources

Repository: [N/A]
Paper: N/A
Demo: [N/A]

Uses

Direct Use

Creative story generation based on given prompts.

Downstream Use

Adaptation to specific genres (e.g., horror, sci-fi, fairy tales).

Out-of-Scope Use

Real-time decision-making in critical domains.
Factual knowledge retrieval.

Bias, Risks, and Limitations

Known limitations:
- GPT-2 Medium has fewer parameters compared to larger LMs, so logical reasoning ability remains limited even after fine-tuning.
- Generated text may contain factual inaccuracies or hallucinations.
- May produce biased or inappropriate outputs due to biases in the training data.

Recommendations

Users should review generated outputs before deployment in any real-world application.

PEFT (Parameter-Efficient Fine-Tuning)

Parameter-Efficient Fine-Tuning is a training approach designed to adapt large pre-trained models to new tasks without updating all their parameters. Instead of fine-tuning the full model, PEFT methods modify only a small set of additional parameters, keeping the rest of the model frozen. This reduces computational requirements, speeds up training, and allows the same base model to be adapted to many tasks efficiently.

LoRA (Low-Rank Adaptation)

Low-Rank Adaptation is a PEFT technique that inserts small, trainable rank-decomposition matrices into the transformer's architecture. Instead of updating the full weight matrices, LoRA learns a low-rank approximation of the weight updates while keeping the original model weights frozen.

Advantages of LoRA include:

Reduced training cost.
Lower VRAM usage.
No need to store a full copy of the fine-tuned model — only adapter weights are saved.
Easy sharing and merging with other adapters.

How to Get Started with the Model

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model_name = "path_or_repo_id"  # Replace with your repo path or Hugging Face model ID
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name)

prompt = "Once upon a time in a distant galaxy,"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=200, temperature=0.8, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Dataset: BookCorpus
Description: A large corpus of books used for language model pretraining, providing diverse narrative text.

Training Procedure

Preprocessing: Tokenized with GPT-2 tokenizer, applied LoRA adapters via PEFT.
Training regime: fp16 mixed precision.
Hardware: 2× NVIDIA RTX A6000 GPUs
Hours used: [N/A]
Cloud Provider: On-premise college GPU cluster.
Compute Region: Belagavi, India.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator.

Hardware Type: 2× NVIDIA RTX A6000 series GPUs
Hours used: [N/A]
Cloud Provider: On-prem (college lab)
Compute Region: India
Carbon Emitted: [N/A]

Technical Specifications

Architecture: Transformer-based GPT-2 Medium with LoRA adapters via PEFT.
Frameworks: Hugging Face Transformers, PEFT 0.14.0, PyTorch.
Precision: fp16 mixed precision.

Citation

BibTeX:

[Add citation if applicable]

APA: