Chocolatine-2-4B-Instruct-DPO-v2.1 is a post-trained version of Qwen/Qwen3-4B-Instruct-2507, designed to improve instruction-following, reasoning, and overall performance in French, while preserving strong multilingual capabilities.
In my evaluation setup, it delivers consistent gains across the tested French benchmarks, pointing to a broad improvement in French capabilities.
Although the post-training pipeline focuses on French preference data, no degradation is observed on English tasks, and slight improvements are sometimes seen, suggesting positive cross-lingual transfer.
Optimized variants (MLX, GGUF) are also available, making the model particularly suitable for local inference.

Note: This model supports only non-thinking mode and does not generate <think></think> blocks in its outputs.
This design is consistent with the goals of the post-training setup, which favors a compact dense instruct model focused on direct generation efficiency and practical downstream use.
For use cases requiring explicit reasoning traces or structured thinking outputs, Qwen/Qwen3.5-4B (thinking mode) is recommended.

Ollama : In addition to the Hugging Face release, quantized 4-bit and 8-bit variants are also available here on Ollama for convenient local inference.

The results indicate a consistent improvement across the tested French benchmarks, covering several capability types. This suggests a broad gain in French performance, while English results remain overall stable.

FR-MT-Bench evaluation is performed on MT-Bench-French, using multilingual-mt-bench with OpenAI/GPT-5 as the LLM judge.
global_mmlu_fr, xwinograd_fr and french_bench results were obtained using EleutherAI LM Eval Harness in a 0-shot evaluation setting.
gpqa-fr:diamond using LightEval/vLLM via kurakurai/Luth process eval.

English benchmark results were obtained using EleutherAI LM Eval Harness in a 0-shot evaluation setting.

Chocolatine-2-4B-Instruct-DPO-v2.1 is derived from Qwen/Qwen3-4B-Instruct-2507 using a multi-step post-training pipeline:

Direct Preference Optimization (DPO) on a DPO-adapted version of Compar:IA data, derived from the preference dataset comparia-votes, part of a public initiative led by the Ministry of Culture (French gov). Previous iterations of the Chocolatine model series also were selected as part of this initiative.
I constructed an original DPO dataset from these votes by transforming them into preference pairs (chosen / rejected), with additional filtering and formatting steps to make them suitable for DPO fine-tuning.
Two dataset variants were created (6k and 13k preference pairs).
The 6k variant was used for the DPO training reported in this release.

A second DPO stage using a french-version of ORCA preference pairs, based on the dataset jpacifico/french-orca-dpo-pairs-revised, commonly used in the Chocolatine training pipeline.
This stage further improves : general instruction alignment, robustness across tasks, cross-lingual capabilities.

The resulting checkpoints were merged using MergeKit with the TIES method.

TIES merging: selects task-relevant parameter updates, reduces destructive interference between models and preserves base model stability.

MergeKit configuration:

The following contains a code snippet illustrating how to use the model generate content based on given inputs.

The Chocolatine-2 model series is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance. It does not have any moderation mechanism.

Developed by: Jonathan Pacifico, 2026
Model type: LLM
Language(s) (NLP): French, English
License: Apache-2.0

Made with ❤️ in France