This repository contains a fine-tuned version of the LiquidAI/LFM2-2.6B model, which has been aligned with French user preferences (available in the comparia-votes dataset) using Direct Preference Optimization (DPO).

The model was a first experiment and so was fine-tuned on a single epoch. Key training metrics are as follows:

You can load and use this model with the Hugging Face transformers library or vllm, for more info follow the vanilla model running guide.