from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained('andersoncliffb/abstracts_to_tweet_model', revision=None) # Load tokenizer
model = AutoModelForSeq2SeqLM.from_pretrained('andersoncliffb/abstracts_to_tweet_model', revision=None) # Load model
pipe = pipeline('text2text-generation', model=model, tokenizer=tokenizer, pad_token_id=tokenizer.pad_token_id)

inputs = ['In this paper, we introduce a novel learning framework for addressing inconsistencies and incompleteness inEffects of Pre-training Task Structure on Cross-lingual Transferof large-scale, multilingual machine reading comprehension (MRC) models. Our proposed method, termed Structured-MRC, employs a new task structure that strategically balances knowledge transfer and specialized information acquisition across languages. Rather than using one universal pre-training task, Structured-MRC synchronizes task-wise pre-training across related language pairs. This technique allows our models to effectively learn and transfer recurring patterns while avoiding overgeneralization. Comprehensive experiments are carried out on eight diverse languages from the XNLI, XNLG, MARC, and WikiMRC datasets, demonstrating that the Structured-MRC framework significantly outperforms state-of-the-art approaches in terms of consistency, comprehensibility, and generality. The insights gained from this study highlight the importance of structuring learning tasks for cross-lingual transfer in MRC, with implications for various NLP applications.']
print(pipe(inputs, max_length=512, do_sample=False))

This model was trained with a synthetic dataset with DataDreamer 🤖💤. The synthetic dataset card and model card can be found here. The training arguments can be found here.