from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "5456es/implicit_reward_Llama-3.2-3B-Instruct_prune_0.3-sigmoid"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example usage
prompt = "Your prompt here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data

Base Model: Llama-3.2-3B-Instruct
Training Method: implicit
Pruning Ratio: unknown
Training Date: 2025-09-15

This model was trained on preference data using the DPO algorithm.

Limitations

This model inherits the limitations of its base model and may have additional limitations due to the pruning process.

Citation

If you use this model, please cite the original DPO paper and the base model.