Cour de Cassation automatic titrage prediction model
Model for the automatic prediction of titrages (keyword sequence) from sommaires (synthesis of legal cases). The models are described in this paper. If you use this model, please cite our research paper (see below).
Model description
The model is a transformer-base model trained on parallel data (sommaires-titrages) provided by the Cour de Cassation. The model was intially trained using the Fairseq toolkit, converted to HuggingFace and then fine-tuned on the original training data to smooth out minor differences that arose during the conversion process. Tokenisation is performed using a SentencePiece model, the BPE strategy and a vocab size of 8000.
Intended uses & limitations
This model is to be used to produce titrages for those sommaires that do not have them or to complement existing (manually) created titrages.
How to use
Model input is the matière (matter) concatenated to the text from the sommaire separated by the token <t>. Each example should be on a single line. E.g. bail <t> La recommendation du tribunal selon l'article... (fictive example for illustrative purposes. The maximum input length of the model is 1024 input tokens (after tokenisation).
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokeniser = AutoTokenizer.from_pretrained("rbawden/CCASS-auto-titrages-base")
model = AutoModelForSeq2SeqLM.from_pretrained("rbawden/CCASS-auto-titrages-base")
matiere = "matter"
sommaire = "full text from the sommaire on a single line"
inputs = tokeniser([matiere + " <t> " + sommaire], return_tensors='pt')
outputs = model.generate(inputs['input_ids'])
tokeniser.batch_decode(outputs, skip_special_tokens=True, clean_up_tokenisation_spaces=True)
Limitations and bias
The models' predictions should not be taken as ground-truth titrages and should always be indicated as being automatically generated. They were designed not to be used as such, but to improve search coverage for improved similarity prediction between different cases (the predicted titrages being used to predict the similarity).
The model is not constrained to predict titres that have previously been seen, so this should be taken into account in the deployment of this model as a titrage tool in order to avoid the multiplication of different titres.
Training data
Training data is provided by the Cour de Cassation (the original source being Jurinet data, but with pseudo-anonymisation applied). For training, we use a total of 159,836 parallel examples (each example is a sommaire-titrage pair). Our development data consists of 1,833 held-out examples.
Training procedure
Preprocessing
We use SentencePiece, the BPE strategy and a joint vocabulary of 8000 tokens. This model was converted into the HuggingFace format and integrates a number of normalisation processes (e.g. removing double doubles, apostrophes and quotes, normalisation of different accent formats, lowercasing).
Training
The model was initialised trained using Fairseq until convergence on the development set (according to our customised weighted accuracy measure - please see the paper for more details). The model was then converted to HuggingFace and training continued to smooth out incoherences introduced during the conversion procedure (incompatibilities in the way the SentencePiece and NMT vocabularies are defined, linked to HuggingFace vocabularies being necessarily the same as the tokeniser vocabulary, a constraint that is not imposed in Fairseq).
Evaluation results
Full results for the initial Fairseq models can be found in the paper.
Results on this converted model coming soon!
BibTex entry and citation info
If you use this work, please cite the following article:
@inproceedings{charmet-et-al-2022-complex,
tite = {Complex Labelling and Similarity Prediction in Legal Texts: Automatic Analysis of France’s Court of Cassation Rulings},
author = {Charmet, Thibault and Cherichi, Inès and Allain, Matthieu and Czerwinska, Urszula and Fouret, Amaury, and Sagot, Benoît and Bawden, Rachel},
booktitle = {Proceedings of the 13th Language Resources and Evaluation Conference},
year = {2022},
address = {Marseille, France},
pages = {4754--4766},
url = {http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.509.pdf}
