⚠️ Note: This model has been re-uploaded to a new organization as part of a consolidated collection.
The updated version is available at https://huggingface.co/aieng-lab/bert-base-cased-mamut.
Please refer to the new repository for future updates, documentation, and related models.

MAMUT Bert (Mathematical Structure Aware BERT)

Model Details

Model Description

• Mathematical Formulas (MF): Masked Language Modeling (MLM) task on math formulas written in LaTeX
• Mathematical Texts (MT): MLM task on mathematical texts (i.e., texts containing LaTeX formulas). The masked tokens are more likely to be a one of the formula tokens or mathematical words (e.g., sum, one, ...)
• Named Math Formulas (NMF): Next-Sentence-Prediction (NSP)-like task associating a name of a well known mathematical identity (e.g., Pythagorean Theorem) with a formula representation (and the task is to classify whether the formula matches the identity described by the name)
• Math Formula Retrieval (MFR): NSP-like task associating two formulas (and the task is to decide whether both describe the same mathematical concept(identity))

• Further pretrained from model: bert-base-cased

Model Sources [optional]

• Repository: aieng-lab/transformer-math-pretraining](https://github.com/aieng-lab/transformer-math-pretraining)
• Paper: MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training

Uses

How to Get Started with the Model

Training Details

Training Data

Training Procedure

Evaluation

Testing Data, Factors & Metrics

Testing Data

Factors

Metrics

Results

Summary

Environmental Impact

• Hardware Type: 8xA100
• Hours used: 48
• Compute Region: Germany

Citation