The MoE Transformer Model Zoos consist of two datasets: AGNews-MoEs and MNIST-MoEs. These datasets contain small-scale Mixture-of-Experts (MoE) Transformer weights and are designed to support research on metanetworks for predicting model generalization directly from internal weight structures.
The dataset is provided as a data.zip file. After extraction, the directory structure is organized as follows:
data/
├── ag_news/
│ ├──… See the full description on the dataset page:
https://huggingface.co/datasets/JohnDoe4765/MoE-Transformer-Model-Zoos.
