You can deploy this model and start earning money today!

AshishKJain/
gpt2-large-alpaca-sft

Name: gpt2-large-alpaca-sft
Rating: 2.5 (8 reviews)

safetensors

gpt2

text-generation

instruction-following

alpaca

openai-community/gpt2-large

finetune

mit

Views

No views yet

GPT2-Large Fine-tuned on Alpaca Instructions

Model Description

This is a GPT2-Large (774M parameters) model fine-tuned on the Stanford Alpaca instruction dataset. The model has been trained to follow instructions and generate appropriate responses in an instruction-following format.

Model Details

Base Model: GPT2-Large (774M parameters)
Training Data: Stanford Alpaca dataset (52,002 instruction-response pairs)
Fine-tuning Method: Supervised Fine-tuning (SFT)
Model Architecture:
- Layers: 36
- Hidden Size: 1280
- Attention Heads: 20
- Context Length: 1024
- Vocabulary Size: 50257

Training Details

Training Split: 85% training, 10% test, 5% validation
Prompt Format: Enhanced Alpaca format
Optimizer: AdamW
Mixed Precision: BF16
Context Window: 1024 tokens
Accuracy on Alpaca test set: 30-40

Usage

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load model and tokenizer
model = GPT2LMHeadModel.from_pretrained("AshishKJain/gpt2-large-alpaca-sft")
tokenizer = GPT2Tokenizer.from_pretrained("AshishKJain/gpt2-large-alpaca-sft")

# Prepare prompt in Alpaca format
prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Give three tips for staying healthy.
### Response:
"""

# Generate response
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    inputs['input_ids'],
    max_new_tokens=256,
    temperature=0.7,
    top_k=50,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id,
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Prompt Format

The model expects prompts in the following format:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
[Your instruction here]
### Response:

For tasks with additional input context:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
[Your instruction here]
### Input:
[Additional context]
### Response:

Limitations and Biases

This model inherits biases from both GPT2-Large and the Alpaca dataset
The Alpaca dataset was generated using GPT-3.5, which may have limitations
The model may generate incorrect or biased information
Context window is limited to 1024 tokens

License

This model is released under the MIT License. However, note that:

GPT2 is released under the Modified MIT License
Alpaca dataset has specific usage guidelines from Stanford

Citation

If you use this model, please cite:

bibtex

@misc{gpt2-large-alpaca,
  author = {Ashish K Jain},
  title = {GPT2-Large Fine-tuned on Alpaca Instructions},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/AshishKJain/gpt2-large-alpaca-sft}}
}

Acknowledgments

OpenAI for GPT2
Stanford for the Alpaca dataset
HuggingFace for the transformers library