Rollout-finetuned version of the BioAnalyst Foundation Model (BFM) Large (702M params) on BioCube monthly climate + biodiversity data using the paper's BFMRollout pipeline (arXiv:2507.09080v2, Section 3.2).

Model Details

Base model: BFM Large (embed_dim=512, depth=10, Swin 3D backbone, 702M params)
PEFT: VeRA (rank=256, d_initial=0.1) — only 63.5K trainable params (0.009%)
Finetuning data: BioCube (28 species, 0.125 deg, monthly, Europe)
Rollout: 12-step push-forward with TBPTT-2 (gradients on last step only)
Training: 50 epochs, AdamW (lr=5e-5, wd=5e-7), bf16-mixed
Best val_loss: 0.627335 (epoch 36)
Architecture: Full BFM pipeline (Encoder → Swin Backbone → Decoder), encoder/decoder frozen

Evaluation on BioCube 28 Species (48 test pairs)

Metric	Rollout FT	Pretrained	Paper
F1 (Eq.20)	0.6446	0.6450	0.9964
Sorensen (Eq.22)	0.0112	0.0110	0.31
Species MAE (norm)	0.020769	0.0210	0.0836

Key Findings

VeRA with only 63.5K trainable parameters (0.009% of 702M) is insufficient for significant improvement over the pretrained baseline on BioCube 28 species
The paper's results (F1=0.9964, S=0.31) likely used a much larger trainable parameter budget or different training setup (e.g., full finetune or higher-rank PEFT)
Species predictions are mostly negative (mean=-0.019, positive_frac=3.3%), indicating the model learned to suppress species signals rather than enhance them

Usage

python

import torch
from safetensors.torch import load_file
from bfm_model.bfm.model import BFMRollout

# 1. Build BFMRollout with VeRA
model = BFMRollout(
    embed_dim=512, depth=10, patch_size=8,
    swin_backbone_size="large", perceiver_latents=16100,
    rollout_steps=1, finetune_mode="peft",
    use_vera=True, peft_r=256, d_initial=0.1,
    peft_mode="single", peft_steps=1,
    # ... (see finetune_rollout_large.py for full config)
)

# 2. Load pretrained base weights
state = load_file("bfm-pretrain-large.safetensors", device="cpu")
model.load_state_dict(state, strict=False)

# 3. Load finetuned VeRA weights from Lightning checkpoint
ckpt = torch.load("epoch=036-val_loss=0.627335.ckpt", map_location="cpu")
model.load_state_dict(ckpt["state_dict"], strict=False)
model.eval()

Files

epoch=036-val_loss=0.627335.ckpt - Best Lightning checkpoint (epoch 36)
last.ckpt - Final checkpoint (epoch 49)
bfm_rollout_eval_results.json - Full evaluation results
finetune_rollout_large.py - Finetuning script
eval_rollout_large.py - Evaluation script

Citation

bibtex

@article{foerster2025bfm,
  title={A Foundation Model for Forecasting Biodiversity Dynamics},
  author={Foerster et al.},
  journal={arXiv preprint arXiv:2507.09080v2},
  year={2025}
}

YoshimuraHiroto/bfm-rollout-finetuned-large

BFM Rollout Finetuned Large (BioCube, VeRA)