You can deploy this model and start earning money today!

haejin1320/
recording-nudge-fusion

Name: recording-nudge-fusion
Availability: InStock

Views

No views yet

Recording Nudge - Late Fusion MLP

Multimodal late-fusion classifier combining audio (YAMNet) and visual (MobileNetV2) logits for 3-class scene classification on Android.

Model Details

Architecture: Dense(6->16, relu) -> Dropout(0.2) -> Dense(16->3)
Input: concatenated logits float32[1, 6] = [audio_logits[3], visual_logits[3]]
Output: fusion_logits float32[1, 3] for [concert, windy_outdoor, quiet_indoor]
Parameters: ~115 (extremely lightweight)
Framework: TF/Keras -> TFLite (direct, no ONNX)
Training data: YouTube genuine pairs + synthetic same-class pairs

Performance (Test Set)

Model	Accuracy
YAMNet alone	0.898
MobileNetV2 alone	0.367
Fusion MLP	0.893

Full Pipeline

Audio (16kHz waveform) -> YAMNet TFLite -> audio_logits[3]  --+
                                                               +--> FusionMLP TFLite -> scene_class
Image (224x224 frame)  -> MobileNetV2 TFLite -> visual_logits[3] -+

Android Usage (TFLite)

See a4_inference_example.py in the training repo for complete inference code.

Classes

Index	Name
0	concert
1	windy_outdoor
2	quiet_indoor

haejin1320/recording-nudge-fusion