You can deploy this model and start earning money today!

nowefnyieryfner/
gpt-10m-dpo-sharegpt

Name: gpt-10m-dpo-sharegpt
Availability: InStock

Views

No views yet

GPT-10M-DPO-ShareGPT

A tiny 10-million parameter GPT-style model trained from scratch, fine-tuned using Supervised Fine-Tuning (SFT) on ShareGPT conversations, and aligned via Direct Preference Optimization (DPO) using preferences judged by Gemini 3.1 Flash Lite.

Model Details

Architecture: Decoder-only Transformer (GPT)
Parameters: ~10 Million
Layers: 6
Heads: 6
Embedding Dimension: 384
Context Length: 512 tokens
Vocabulary Size: 50,257 (custom BPE tokenizer)

Training Stages

Pretraining: Continued pretraining for ~24.6M tokens (part of a full epoch over a 95M curated flat-packed token dataset consisting of 50% code, 20% knowledge, and 10% textbooks).
Supervised Fine-Tuning (SFT): Fine-tuned for 500 steps using float16 precision on 10,000 multi-turn conversations from the ShareGPT dataset. Best SFT validation loss reached 3.4184.
Direct Preference Optimization (DPO): Aligned on 255 preference pairs. Prompts were sampled from SFT logs, completions were generated by the SFT model, and preferred responses were judged by Gemini 3.1 Flash Lite.

How to Load and Sample

You can use the custom PyTorch definition in model.py and tokenizer.json to load the model:

python

import torch
from tokenizers import Tokenizer
from model import GPT

# Load model checkpoint
checkpoint = torch.load("dpo_model.pt", map_location="cpu")
config = checkpoint["config"]
model = GPT(config)
model.load_state_dict(checkpoint["model"])
model.eval()

# Load tokenizer
tokenizer = Tokenizer.from_file("tokenizer.json")

# Generate text
prompt = "<|im_start|>user\nWrite a python function to check if a number is prime.<|im_end|>\n<|im_start|>assistant\n"
x = torch.tensor(tokenizer.encode(prompt).ids, dtype=torch.long).unsqueeze(0)
y = model.generate(x, max_new_tokens=150, temperature=0.8, top_k=50)
print(tokenizer.decode(y[0].tolist()))

nowefnyieryfner/gpt-10m-dpo-sharegpt

GPT-10M-DPO-ShareGPT

Model Details

Training Stages

How to Load and Sample

nowefnyieryfner/
gpt-10m-dpo-sharegpt