gustavecortal/

gpt-j-fr-covid-news

This is a version of Cedille's GPT-J (fr-boris) with 6 billion parameters fine-tuned on COVID-19 French News dataset to generate French headlines related to COVID-19.

You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by GPT-J 8bit.

Here's how to run it:

This model can be easily loaded using the GPTJForCausalLM functionality:

Remember, you have to Monkey-Patch the model before loading it (see Colab above).

How not to be disoriented in a pandemic era when faced with an immense flow of information? This page features one thousand AI-generated French headlines related to COVID-19.

Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the mesh-transformer-jax codebase.

Boris was trained on around 78B tokens of French text from the C4 dataset.