Goswami et al., 2024. "MOMENT: A Family of Open Time-series Foundation Models." Proceedings of the 41st International Conference on Machine Learning (ICML).

HuggingFace: small · base · large

Package: momentfm==0.1.4 (pip-installable, no code submodule)

GitHub: https://github.com/moment-timeseries-foundation-model/moment

Each size is pinned to a specific HF commit (see moment_prep/config.py) so the export is fully reproducible regardless of upstream changes to main.

MIT (Auton Lab, Carnegie Mellon University)

MOMENT is a family of T5-based time-series foundation models pre-trained with a masked-reconstruction objective on the Time-series Pile. The input is split into non-overlapping patches of 8 observations, normalized with reversible instance normalization (RevIN), embedded with a linear patch projection plus a learned positional embedding, and encoded by a Flan-T5 encoder stack. We export the encoder in "embedding" mode: it produces one embedding per patch, which is masked-mean-pooled over the valid patches to yield a single fixed-size vector. The three sizes differ only in width:

All three share an identical ONNX interface and preprocessing.

Unlike the Chronos family, MOMENT has a fixed context window of 512 observations (64 patches of 8); the sequence length is not a dynamic axis. Cap each light curve to its last 512 observations and left-pad shorter curves to 512 with NaN.

MOMENT's architecture has no timestamp input — embed() takes only the value tensor x_enc and a boolean input_mask; positional information is purely index-based (a sinusoidal embedding over patch index), and the model was pre-trained on regularly sampled series. There is therefore no way to feed modified Julian dates directly.

This export follows the same timestamp-free convention this repo uses for the Chronos models: light curves are treated as equally spaced in observation order, and NaN marks padded / missing positions. (That convention comes from the StarEmbed benchmark, which evaluates Chronos and Astromer but not MOMENT — we extend it to MOMENT for consistency.)

If you instead want the model to "see" real time, you must do it in pre-processing: resample / bin the light curve onto a uniform 512-step time grid so that patch index corresponds to elapsed time, marking empty bins with NaN. The ONNX interface supports both approaches unchanged — only how you fill the 512-step context differs.

batch is dynamic; the sequence length is fixed at 512. A patch of 8 observations contributes to the output only if all 8 of its positions are valid (non-NaN), matching MOMENT's patch-level masking.

One file per size, moment1-<size>.onnx, with two named outputs:

RevIN instance normalisation (masked mean subtraction, std scaling) is applied internally by the model.

Source: AutonLab/MOMENT-1-{small,base,large} (loaded automatically via MOMENTPipeline.from_pretrained, each pinned to a specific revision). Dataset: the Time-series Pile (AutonLab/Timeseries-PILE), a large collection of public time series spanning many domains.