Model	Initialised from...	Training Data
jargon-general-base	scratch	8.5GB Web Corpus
jargon-general-biomed	jargon-general-base	5.4GB Medical Corpus
jargon-general-legal	jargon-general-base	18GB Legal Corpus
jargon-multidomain-base	jargon-general-base	Medical+Legal Corpora
jargon-legal	scratch	18GB Legal Corpus
jargon-legal-4096 (this model)	scratch	18GB Legal Corpus
jargon-biomed	scratch	5.4GB Medical Corpus
jargon-biomed-4096	scratch	5.4GB Medical Corpus
jargon-NACHOS	scratch	NACHOS
jargon-NACHOS-4096	scratch	NACHOS

Evaluation

The Jargon models were evaluated on an range of specialized downstream tasks.

Legal Domain Benchmark

Results averaged across five runs with varying random seeds.

	ECtHR-FR	OACS	SJP
Task Type	Document Classification	Document Classification	Document Classification
Metric	Macro-F1	Macro-F1	Macro-F1
jargon-general-base	42.9	50.8	55.1
jargon-multidomain-base	44.5	55.6	58.1
jargon-general-legal	43.1	49.9	54.5
jargon-legal	44.6	51.6	56.7
jargon-legal-4096	45.9	54.1	68.2

For more info please check out the paper, accepted for publication at LREC-COLING 2024.

Using Jargon models with HuggingFace transformers

You can get started with jargon-legal-4096 using the code snippet below:

python

from transformers import AutoModelForMaskedLM, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("PantagrueLLM/jargon-legal-4096", trust_remote_code=True)
model = AutoModelForMaskedLM.from_pretrained("PantagrueLLM/jargon-legal-4096", trust_remote_code=True)

jargon_maskfiller = pipeline("fill-mask", model=model, tokenizer=tokenizer)
output = jargon_maskfiller("Il est allé au <mask> hier")

You can also use the classes AutoModel, AutoModelForSequenceClassification, or AutoModelForTokenClassification to load Jargon models, depending on the downstream task in question.

Language(s): French
License: MIT
Developed by: Vincent Segonne
Funded by
- GENCI-IDRIS (Grant 2022 A0131013801)
- French National Research Agency: Pantagruel grant ANR-23-IAS1-0001
- MIAI@Grenoble Alpes ANR-19-P3IA-0003
- PROPICTO ANR-20-CE93-0005
- Lawbot ANR-20-CE38-0013
- Swiss National Science Foundation (grant PROPICTO N°197864)
Authors
- Vincent Segonne
- Aidan Mannion
- Laura Cristina Alonzo Canul
- Alexandre Audibert
- Xingyu Liu
- Cécile Macaire
- Adrien Pupier
- Yongxin Zhou
- Mathilde Aguiar
- Felix Herron
- Magali Norré
- Massih-Reza Amini
- Pierrette Bouillon
- Iris Eshkol-Taravella
- Emmanuelle Esperança-Rodier
- Thomas François
- Lorraine Goeuriot
- Jérôme Goulian
- Mathieu Lafourcade
- Benjamin Lecouteux
- François Portet
- Fabien Ringeval
- Vincent Vandeghinste
- Maximin Coavoux
- Marco Dinarelli
- Didier Schwab

Citation

If you use this model for your own research work, please cite as follows:

bibtex

@inproceedings{segonne:hal-04535557,
  TITLE = {{Jargon: A Suite of Language Models and Evaluation Tasks for French Specialized Domains}},
  AUTHOR = {Segonne, Vincent and Mannion, Aidan and Alonzo Canul, Laura Cristina and Audibert, Alexandre and Liu, Xingyu and Macaire, C{\'e}cile and Pupier, Adrien and Zhou, Yongxin and Aguiar, Mathilde and Herron, Felix and Norr{\'e}, Magali and Amini, Massih-Reza and Bouillon, Pierrette and Eshkol-Taravella, Iris and Esperan{\c c}a-Rodier, Emmanuelle and Fran{\c c}ois, Thomas and Goeuriot, Lorraine and Goulian, J{\'e}r{\^o}me and Lafourcade, Mathieu and Lecouteux, Benjamin and Portet, Fran{\c c}ois and Ringeval, Fabien and Vandeghinste, Vincent and Coavoux, Maximin and Dinarelli, Marco and Schwab, Didier},
  URL = {https://hal.science/hal-04535557},
  BOOKTITLE = {{LREC-COLING 2024 - Joint International Conference on Computational Linguistics, Language Resources and Evaluation}},
  ADDRESS = {Turin, Italy},
  YEAR = {2024},
  MONTH = May,
  KEYWORDS = {Self-supervised learning ; Pretrained language models ; Evaluation benchmark ; Biomedical document processing ; Legal document processing ; Speech transcription},
  PDF = {https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf},
  HAL_ID = {hal-04535557},
  HAL_VERSION = {v1},
}