This model is a transformer-based encoder-decoder model for multiple NLP tasks:
Text summarization
Translation (English-Arabic)
Question-answering
Model Architecture
Model type: miscovery
Number of parameters: 485674144
Encoder layers: 12
Decoder layers: 12
Attention heads: 12
Hidden size: 768
Feed-forward size: 3072
Training
The model was trained in two stages:
Pre-training on sentence rearrangement tasks
Fine-tuning on downstream tasks
Usage
Install the package:
pip install miscovery-model
Run the model using a script:
python
1from miscovery_model import standard_pipeline
23# Create a pipeline4model = standard_pipeline("miscovery/model")56# Use it7result = model("Translate this to Arabic: What year did World War I begin?")8print(result)
Limitations
This model was trained on specific datasets and may not generalize well to all domains.
