Pensez is a bilingual (French-English) reasoning model designed to maximize efficiency with significantly reduced training data. The model leverages a curated dataset focusing on daily reasoning tasks and scientific questions to enhance performance.
These optimizations result in superior reasoning capabilities while maintaining robust general understanding compared to models like
DeepSeek-R1-Distill-Qwen-7B.
Pensez is built upon
Qwen 2.5 Instruct 7B and trained over five epochs.
Pensez was trained on the hand-curated
Pensez v0.1 dataset containing 2,000 samples (1,000 French, 1,000 English).
Pensez was evaluated on French-specific benchmarks, demonstrating strong reasoning ability and improved task-specific performance:
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4model_path = "HoangHa/Pensez-v0.1-e5"
5
6# Load tokenizer and model
7tokenizer = AutoTokenizer.from_pretrained(model_path)
8model = AutoModelForCausalLM.from_pretrained(
9 model_path, torch_dtype=torch.float16, device_map="auto"
10)
11
12# Example input
13messages = [{"role": "user", "content": "Bonjour!"}]
14input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt').to("cuda")
15
16generated_ids = model.generate(input_ids, max_new_tokens=2500, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
17response = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_space=True)
18print(f"Réponse: {response}")
1@misc{dao2025alphamazeenhancinglargelanguage,
2 title={Pensez: Less Data, Better Reasoning – Rethinking French LLM},
3 author={Ha Huy Hoang},
4 year={2025},
5 archivePrefix={arXiv},
6 primaryClass={cs.CL},
7 url={},
8}
