You can deploy this model and start earning money today!

hyunseop/
vision-cardio-rppg

Name: vision-cardio-rppg
Availability: InStock

physiological-measurement

Views

No views yet

Vision Cardio — rPPG Heart-Rate Model (PhysNet, UBFC fine-tuned)

Camera-based remote photoplethysmography (rPPG) model that estimates heart rate from short front-camera face clips. PhysNet-style 3D-CNN that outputs an rPPG waveform; HR is read off as the FFT peak in the 0.7–3.0 Hz band.

Powers the on-device VISION CARDIO iOS app (offline, Korean UI, wellness-only): front camera → HR, with running HR-phase zones (1–5) and weights set/rest recovery coaching measured against a calibrated baseline HR. Built and running on iPhone (iOS 16+); Core ML runs on GPU/CPU.

Files

file	what
`rppg_physnet_ubfc.pt`	PyTorch checkpoint (PhysNet, width=32), UBFC fine-tuned
`VisionCardioHR.mlpackage`	Core ML model for iOS 16+ (same weights, traced)

Performance

Trained on synthetic SCAMPS, then fine-tuned on real UBFC-rPPG faces with a strict by-participant split (no subject leakage).

stage	HR MAE (vs contact-PPG)
zero-shot (SCAMPS only) on UBFC	5.63 bpm
after UBFC fine-tune (val)	2.80 bpm

Fine-tune split: 40 train / 10 val subjects; best-by-val checkpoint (early-stopped — the tiny set overfits after a couple of epochs).

I/O contract

input  "clip"     : (1, 3, T=128, H=112, W=112) float, RGB, [0,1] normalized
output "waveform" : (1, 128)  predicted rPPG pulse
HR = FFT peak in [0.7, 3.0] Hz, fs = clip_frames / clip_seconds = 128 / 20 = 6.4 Hz

Core ML metadata carries clip_frames, clip_size, clip_seconds, fs, hr_band_hz so the app stays in sync automatically.

Usage (PyTorch)

python

import torch
from ml.physnet import PhysNet, hr_from_wave   # repo: PFSV/ByeongYeok_RnD_NLP (vision_cardio)
ck = torch.load("rppg_physnet_ubfc.pt", map_location="cpu")
m = PhysNet(width=ck["width"]); m.load_state_dict(ck["state_dict"]); m.eval()
wave = m(clip)[0].numpy()           # clip: (1,3,128,112,112) in [0,1]
bpm = hr_from_wave(wave, fs=6.4)

Caveats

Wellness/research only — not a medical device, no diagnostic claims.
Fine-tuned on a small real set (UBFC); generalization to arbitrary lighting / skin tone / motion is not guaranteed. Use confidence gating before trusting an estimate.

Code, pipeline & iOS app: PFSV/ByeongYeok_RnD_NLP, branch vision-cardio-ubfc-app → TUTORIAL_DRILL/neuro-nlp/vision_cardio.

hyunseop/vision-cardio-rppg