library_name: transformers
tags:
- gpt2
- causal-lm
- bilingual
- sentencepiece
- french
- english
pipeline_tag: text-generation
datasets:
- climb-mao/babylm-fra
- elliepreed/l2-corpus-10m
license: other # change to "apache-2.0" or "mit" if that's correct
model-index:
- name: BGPT (French+English) – 128k steps
results: []
BGPT – French + English (GPT-2 style)
Small bilingual GPT-2–style language model trained on French and English with SentencePiece tokenizers.
This model is trained on both French 🇫🇷 and English 🇬🇧, but it does not come with a single AutoTokenizer.
Instead, we provide two SentencePiece tokenizers:
tokenizers/french.modeltokenizers/english.model
You can load either depending on the language you want to work with.
Load the model
from transformers import AutoModelForCausalLM
import torch
model_id = "elliepreed/bgpt-french-english"
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained(model_id).to(device).eval()
Load both tokenizers
import sentencepiece as spm
from huggingface_hub import hf_hub_download
fr_path = hf_hub_download(model_id, "tokenizers/french.model")
en_path = hf_hub_download(model_id, "tokenizers/english.model")
sp_fr = spm.SentencePieceProcessor(model_file=fr_path)
sp_en = spm.SentencePieceProcessor(model_file=en_path)
Example: French generation
prompt = "Paris est"
ids = sp_fr.encode(prompt, out_type=int) + [sp_fr.eos_id()]
input_ids = torch.tensor([ids], device=device)
out = model.generate(
input_ids,
max_new_tokens=40,
do_sample=True,
top_p=0.95,
temperature=0.9,
eos_token_id=sp_fr.eos_id(),
pad_token_id=sp_fr.pad_id(),
)
print("FR:", sp_fr.decode(out[0].tolist()[len(ids):]))
