Llama-100m-DPO-ShareGPT
An aligned, instruction-following Llama-style model based on the 100m parameters base architecture.
Model Details
- Base Model: llama-100m-base
- Features: Rotary Position Embeddings (RoPE), RMSNorm, SwiGLU MLP, No Bias
- Alignment Stages:
- SFT: Fine-tuned on the ShareGPT dataset for 500 steps.
- DPO: Aligned on 255 preference pairs judged by Gemini-1.5-Pro for 100 steps.
Load and Run
You can use the Hugging Face Hub library to run generation dynamically:
# Download and run generate.py from the repository
