from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")
tokenizer = AutoTokenizer.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")

# Generate text (note: this model has random weights!)
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))

Important Notes

Base model: meta-llama/Llama-3.3-70B-Instruct
Seed: 42
Hidden size: 256
Number of layers: 12
Number of attention heads: 4
Vocabulary size: 128256
Max position embeddings: 131072

⚠️ This model has random weights and is not trained! It's designed for:

Testing model loading and inference pipelines
Benchmarking model architecture
Educational purposes
Rapid prototyping where actual model performance isn't needed

The model will generate random/nonsensical text since it hasn't been trained on any data.

Creation

This model was created using the upload_tiny_llama.py script from the minimal-grpo-trainer repository.