You can deploy this model and start earning money today!

Fudan-fMRI/
CineSync

Name: CineSync
Availability: InStock

Views

No views yet

CineBrain: A Large-Scale Multi-Modal Brain Dataset for Naturalistic Audiovisual Narrative Processing

Jianxiong Gao, Yichang Liu, Baofeng Yang, Jianfeng Feng, Yanwei Fu†

ArXiv

Dataset

📘 Overview

CineBrain is a large-scale multimodal brain dataset containing fMRI, EEG, and ECG recordings collected while participants watched episodes of The Big Bang Theory.

It supports research on:

Video decoding from multimodal neural signals
Auditory decoding from naturalistic narrative listening
Cross-modal EEG-to-fMRI translation
Stimulus-to-brain modeling under real-world audiovisual viewing

🧩 Codebase

This codebase is built upon CogVideo (v1.0, not CogVideo 1.5):
👉 https://github.com/zai-org/CogVideo/releases/tag/v1.0

🚀 Quick Start

1️⃣ Install CogVideo Environment

Follow the official CogVideo v1.0 setup guide.

2️⃣ Install Additional Dependencies

pip install -r requirements.txt

3️⃣ Prepare Models

Download the CogVideo-5B base model
Download CineBrain-related weights from HuggingFace

📦 Dataset

Full dataset, documentation, and metadata are available on HuggingFace:

👉 CineBrain Dataset

Includes:

fMRI recordings
EEG & ECG signals
Synchronized video, audio, and subtitles
Time-aligned captions & event annotations

📄 Citation

If you find this work useful, please cite:

bibtex

@article{gao2025cinebrain,
  title={CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing},
  author={Gao, Jianxiong and Liu, Yichang and Yang, Baofeng and Feng, Jianfeng and Fu, Yanwei},
  journal={arXiv preprint arXiv:2503.06940},
  year={2025}
}

Fudan-fMRI/CineSync