You can deploy this model and start earning money today!

quinnmong/
CAS4133-aligned-model

Name: CAS4133-aligned-model
Availability: InStock

pytorch

llama

Views

No views yet

CAS4133-aligned-model

This model is trained as part of CAS4133 Assignment 2 (May 2025), using a Llama 3.2 1B base model fine-tuned with Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) via the Unsloth framework.

Model Description

Base Model: unsloth/Llama-3.2-1B-unsloth-bnb-4bit
Techniques: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO)
Framework: Unsloth + Hugging Face Transformers
Hardware: Trained on 1x NVIDIA RTX 3090

Training Data

A cleaned subset of the Alpaca dataset was used, truncated to 30,000 examples.

Intended Use

This model is optimized for general alignment tasks with natural language instructions. For academic and research purposes only.

Limitations

Trained on a relatively small dataset.
No safety guardrails included by default.

Citation

If you use this model, please cite the CAS4133 coursework guidelines.

quinnmong/CAS4133-aligned-model