Cedille is a project to bring large language models to non-English languages.
Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the
mesh-transformer-jax codebase.
Boris was trained on around 78B tokens of French text from the
C4 dataset. We started training from GPT-J, which has been trained on
The Pile. As a consequence the model still has good performance in English language. Boris makes use of the unmodified GPT-2 tokenizer.
Boris is named after the great French writer
Boris Vian.
For the time being, the easiest way to test the model is to use our
publicly accessible playground.
Cedille is a relatively large model and running it in production can get expensive. Consider contacting us for API access at
hello@cedille.ai.
1@misc{muller2022cedille,
2 title={Cedille: A large autoregressive French language model},
3 author={Martin M{\"{u}}ller and Florian Laurent},
4 year={2022},
5 eprint={2202.03371},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL}
8}
For any custom development please contact us at
hello@cedille.ai.
