Model Card for Llama-3.3-Argunaut-1-70B-SPIN

This model is a fine-tuned version of DebateLabKIT/Llama-3.3-Argunaut-1-70B-SFT. It has been trained using TRL and vLLM.

It is released as part of the

Argunauts Project

Quick start

python

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="DebateLabKIT/Llama-3.3-Argunaut-1-70B-SPIN", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with Self-Play Fine-Tuning (SPIN), a method introduced in Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models.

A general description of the training procedure can be found in this blog post.

For SPIN training we use

Hugging Faces's TRL
Microsoft's DeepSpeed
Cognitive Computation's Spectrum
LinkedIn's Liger Kernels

Training parameters and set-up, same for each epoch:

Parameter	Value
dpo loss	sigmoid
learning rate	2e-7
lr scheduler	linear
effective batch size	64
deepspeed ZeRO	stage 3
deepspeed CPU offload	optimizer & parameters
spectrum top % snr ratios targeted	30%

Training one epoch takes roughly 4 hours on 4 H100.

Framework versions

TRL: 0.14.0
Transformers: 4.46.3
Pytorch: 2.4.1
Datasets: 3.1.0
Tokenizers: 0.20.3

Chat Example


[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]

[object Object]