For internal purposes and for testing, we have made a monolingual paraphrasing model from Sentence Transformers usable for
German + English via
Knowledge Distillation.
The decision was made in favor of
sentence-transformers/paraphrase-distilroberta-base-v2 because this model has no public available multilingual version (to our knowledge).
In addition, it has significantly more training samples compared to its predecessor: 83.3 million samples were used instead of 24.6 million samples.
First we downloaded some german-english parallel datasets via
get_parallel_data_*.py.
Then we started knowledge distillation with
make_multilingual_sys.pyThis work was funded by the European Regional Development Fund (EFRE) and the State of Brandenburg. Project/Vorhaben: "ProFIT: Natürlichsprachliche Dialogassistenten in der Pflege".
