You can deploy this model and start earning money today!

rayondemiel/
freem-modernizer-lstm

Name: freem-modernizer-lstm
Availability: InStock

freem-modernizer-lstm

Views

No views yet

Freem Modernizer LSTM

This model normalizes historical French texts to modern French. This model comes from the work of R. Bawden & Al. and a reworking of the following Transformer pipeline repository

1. Usage

python

from transformers import pipeline

normalizer = pipeline(model="rayondemiel/freem-modernizer-lstm", trust_remote_code=True)

# Single input
result = normalizer("Elle haïſſoit particulierement le Cardinal de Lorraine;")
print(result)

# List of inputs
inputs = ["Elle haïſſoit particulierement le Cardinal de Lorraine;", 
          "Adieu, i'iray chez vous tantoſt vous rendre grace."]
outputs = normalizer(inputs)
print(outputs)

# With CUDA and batch processing
normalizer = pipeline(
    model="rayondemiel/freem-modernizer-lstm", 
    trust_remote_code=True,
    device=0,  # Use first CUDA device
    batch_size=16  # Process 16 sentences at once
)

2. Model Details

Architecture: LSTM
Vocabulary Size: 4000
Training Data: Freem corpus
Tokenizer: SentencePiece
Embedding Size: 384
Hidden Size: 768

3. Model Files

checkpoint_best.pt: Trained model finetuned weights
bpe_joint_4000.model: Sentencepiece model file
data_norm_bin_4000/: fairseq data

4. dependencies

Use with python3.9

pip install torch==2.5.1 --index-url https://download.pytorch.org/whl/cu118


transformers>=?
fairseq==0.12.2
torch==2.5.1
omegaconf==2.0.5
gdown==4.2.0
sentencepiece==0.2.0
numpy==1.25.2

5. Citation

**Rachel Bawden, Jonathan Poinhos, Eleni Kogkitsidou, Philippe Gambette, Benoît Sagot, Simon Gabay** (2022). *Automatic Normalisation of Early Modern French*. In *Proceedings of the Thirteenth Language Resources and Evaluation Conference*, Marseille, France, European Language Resources Association. [Lien](https://aclanthology.org/2022.lrec-1.358/).