You can deploy this model and start earning money today!

course-genai-w24/
week4-phi-1.5-sft-shakespeare

Name: week4-phi-1.5-sft-shakespeare
Availability: InStock

Views

No views yet

Model Card for Model ID

Summary

This is a supervised fine-tuned model for text completion based on Phi 1.5. It has been finetuned on a filtered version of the The Complete Works of William Shakespeare, which can be found and downloaded from here: https://www.gutenberg.org/ebooks/100.

Model Description

Developed by: Course Organizers
Finetuned from model: microsoft/phi-1_5

Training Details

This model has been trained using the TLR library and SFTTrainer class from Huggingface.

Training Data

The Complete Works of William Shakespeare, which can be found and downloaded from here: https://www.gutenberg.org/ebooks/100

Training Hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
per_device_train_batch_size: 4
seed: 42
gradient_accumulation_steps: 1
optimizer: Adam with betas=(0.9, 0.95)
lr_scheduler_type: linear
weight_decay: 0.1
num_epochs: 1

Framework Versions

accelerate==0.26.1
datasets==2.16.1
transformers==4.45.2
trl==0.11.2

Compute Infrastructure and Hardware

Slurm cluster with 8 x H100 Nvidia GPUs.

course-genai-w24/week4-phi-1.5-sft-shakespeare