You can deploy this model and start earning money today!

Jeevaaaa/
image-to-song-recommender

Name: image-to-song-recommender
Availability: InStock

clip

music-recommendation

image-to-music

zero-shot-classification

multimodal

zero-shot-image-classification

mit

Views

No views yet

Image → Song Recommender

Given a photo, this pipeline suggests songs that match its visual mood — similar to Instagram's "suggested audio" feature. Built entirely with free, pretrained components (no custom model training).

How it works

Mood detection (zero-shot): CLIP scores the input image against a fixed set of mood text prompts (e.g. "a happy, energetic, joyful photo", "a calm, peaceful, relaxing photo") and returns a confidence weight per mood.
Mood → audio-feature mapping: each mood is mapped by hand to a target point in Spotify's audio-feature space — valence, energy, danceability, acousticness, tempo. The top-3 detected moods are blended (weighted by CLIP's confidence) into one target vector.
Song matching: a NearestNeighbors search (scikit-learn) over a Spotify tracks dataset finds the songs whose audio features are closest to that target vector.

No part of this pipeline is trained — it's a zero-shot composition of an existing vision-language model with a hand-authored mood→feature mapping and a nearest-neighbor lookup.

Files

pipeline.py — full pipeline: image in, ranked song recommendations out
requirements.txt — dependencies

Usage

python

from pipeline import recommend_songs
from PIL import Image

image = Image.open("your_photo.jpg")
mood_breakdown, songs = recommend_songs(image, n_songs=5)

for mood, weight in mood_breakdown:
    print(f"{mood}: {weight:.2f}")

for song in songs:
    print(song["track_name"], "-", song["artist"])

Note: pipeline.py expects a spotify_dataset.csv file in the same directory (not included in this repo — see Dataset section below) with columns: track_name, artists, valence, energy, danceability, acousticness, tempo, and optionally track_genre.

Dataset

Song matching is powered by the Spotify Tracks Dataset (114k tracks, CC0 license). Not redistributed here — download separately and place as spotify_dataset.csv alongside pipeline.py.

Limitations

Mood categories are a fixed, hand-authored list (12 moods) — images outside these categories will be forced into the closest available mood.
The mood → audio-feature mapping is a manual design choice, not learned from data.
CLIP was trained on general image-text pairs, not specifically on aesthetic/mood classification, so mood detection is approximate.

License

MIT (this pipeline code). CLIP itself is licensed separately by OpenAI/Hugging Face.

Jeevaaaa/image-to-song-recommender