gpt2-large-finetuned-wikitext103 – AI Model by Rubin-Wei | AlphaNeural AI

You can deploy this model and start earning money today!

Rubin-Wei/
gpt2-large-finetuned-wikitext103

like

0

openai-community/gpt2-xl

Views

No views yet

GPT2-large-Finetuned-WikiText103

Model Description

A fine-tuned version of GPT2-large on the WikiText-103 dataset.

Paper: MLP Memory: A Retriever-Pretrained Memory for Large Language Models
GitHub: https://github.com/Rubin-Wei/MLPMemory
Base Model: GPT2-large
Training Dataset: WikiText-103
Model Size: 744M parameters

Performance on WikiText-103

Model	Perplexity	Improvement
GPT2-large (baseline)	15.80	-
GPT2-large-Finetuned	10.42	-5.38

Training Details

Training Data: WikiText-103 (103M tokens)
Optimizer: AdamW
Learning Rate: 2e-5 with cosine schedule

Citation

This model was released as part of the paper "MLP Memory: A Retriever-Pretrained Memory for Large Language Models".

For more information, see: https://github.com/Binn0/MLPMemory.

If you use this model, please cite:

bibtex

@inproceedings{Wei2025MLPMA,
  title={MLP Memory: A Retriever-Pretrained Memory for Large Language Models},
  author={Rubin Wei and Jiaqi Cao and Jiarui Wang and Jushi Kai and Qipeng Guo and Bowen Zhou and Zhouhan Lin},
  year={2025},
  url={https://api.semanticscholar.org/CorpusID:281658735}
}