You can deploy this model and start earning money today!

mia-project-2025/
pythia-160M-finetuned-writing-prompts

Name: pythia-160M-finetuned-writing-prompts
Availability: InStock

safetensors

apache-2.0

Views

No views yet

Pythia-160M Fine-Tuned on WritingPrompts

This repository contains a fine-tuned version of EleutherAI’s Pythia-160M model on the WritingPrompts dataset.
The goal is to adapt the base model for story and creative text generation.

Model Details

Base model: EleutherAI/pythia-160m
Dataset: euclaise/writingprompts
Task: Causal Language Modeling (CLM)
Sequence length: 1024 tokens
Precision: Mixed (FP16)
Training objective: Next-token prediction

Training Setup

Batch size (per device): 8
Gradient accumulation steps: 2
Effective batch size: 16
Learning rate: 1e-6 (cosine decay scheduler, warmup=500 steps, warmup_ratio=0.03)
Weight decay: 0.1
Epochs: 10
Max gradient norm: 1.0
Frameworks: Hugging Face Transformers + PyTorch
Hardware: CUDA (multi-GPU setup possible)

Results

Training Summary:

json

{
  "train_runtime": 359.4431,
  "train_samples_per_second": 222.455,
  "train_steps_per_second": 3.478,
  "train_loss": 3.2353,
  "epoch": 10.0
}


## Final Evaluation Metrics (Epoch 10):

Metric	Value
eval_loss	3.1341
eval_perplexity	22.9670
eval_runtime	1.8666
eval_samples_per_second	251.2600
eval_steps_per_second	8.0360
epoch	10.0

mia-project-2025/pythia-160M-finetuned-writing-prompts

Pythia-160M Fine-Tuned on WritingPrompts

Model Details

Training Setup

Results

mia-project-2025/
pythia-160M-finetuned-writing-prompts