Scene Classifier - ResNet18
This model is a fine-tuned version of ResNet-18 pretrained on ImageNet. It classifies images into 4 scene categories: cafe, gym, library, and outdoor.
Model description
This model uses a dataset of video frames extracted from recordings of different indoor and outdoor locations. The ResNet-18 architecture was chosen for its balance of accuracy and computational efficiency, using transfer learning from ImageNet pretrained weights. Only the final fully-connected layer was retrained for the 4-class classification task.
The model is part of a larger pipeline that generates contextual music based on scene classification combined with weather and temporal metadata.
Intended uses & limitations
Intended use:
- Scene classification for context-aware applications
- Image-to-music generation pipelines
- Indoor/outdoor scene detection
- Educational demonstrations of transfer learning
Limitations:
- Limited to 4 specific scene categories (cafe, gym, library, outdoor)
- Limited to Carnegie Mellon University (CMU) campus
- Trained on relatively small dataset extracted from videos
- May not generalize well to significantly different scene compositions
- Performance may degrade on low-quality or heavily edited images
- Indoor scenes may be confused if they share similar visual features
Training and evaluation data
Dataset: Dataset used is: madhavkarthi/project-1-location-classification-dataset
Video frame extraction from 4 scene categories
- Classes: cafe, gym, library, outdoor
- Source: Personal video recordings of various locations
- Extraction: Sampled every 10th frame from videos
- Total frames: Approximately 500+ images
- Format: JPEG, 224x224 resolution after preprocessing
The dataset represents real-world indoor and outdoor environments with varying lighting conditions, angles, and compositions.
Training procedure
Data preprocessing
Images were preprocessed with resize to 224x224 and converted to tensors.
Model architecture
- Base model: ResNet-18 (ImageNet pretrained)
- Modified layer: Final fully-connected layer changed from 1000 classes to 4 classes
- Transfer learning: All layers except final FC layer retained pretrained weights
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-4
- train_batch_size: 32
- optimizer: Adam with default betas=(0.9, 0.999)
- loss_function: CrossEntropyLoss
- num_epochs: 3
- device: CUDA (GPU accelerated)
Training results
Training was conducted over 3 epochs with consistent loss reduction:
| Epoch | Training Loss | Status |
|---|
| 1 | 0.3395 | ✓ |
| 2 | 0.0111 | ✓ |
| 3 | 0.0041 | ✓ |
Note: Formal validation metrics were not computed during training. Model was validated qualitatively on held-out images.
Usage
This can be used to classify any input image into one of four classifiers: Library, Cafe, Gym, Outdoor.
Loading the model
import torch
from torchvision import models, transforms
from PIL import Image
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = models.resnet18(weights=None)
model.fc = torch.nn.Linear(model.fc.in_features, 4)
model.load_state_dict(torch.load("pytorch_model.pth", map_location=device))
model = model.to(device)
model.eval()
transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
])
class_labels = ["cafe", "gym", "library", "outdoor"]
Inference
image = Image.open("your_image.jpg")
if image.mode != 'RGB':
image = image.convert('RGB')
input_tensor = transform(image).unsqueeze(0).to(device)
with torch.no_grad():
outputs = model(input_tensor)
predicted_idx = outputs.argmax(dim=1).item()
predicted_class = class_labels[predicted_idx]
confidence = torch.softmax(outputs, dim=1)[0][predicted_idx].item()
print(f"Predicted: {predicted_class} (confidence: {confidence:.2%})")
Framework versions
- PyTorch: 2.0+
- Torchvision: 0.15+
- Python: 3.8+
- Pillow: 9.0+
Model Architecture Details
ResNet-18 Structure:
- Input: 3x224x224 RGB image
- Convolutional layers with residual connections
- Global average pooling
- Final FC layer: 512 to 4 classes
- Total parameters: approximately 11.7M (only approximately 2K trainable in final layer)
Additional Information
This model was developed as part of a course project (24-679) exploring multimodal AI systems.
It serves as the visual classification component in an image-to-music generation pipeline that combines scene recognition,
metadata extraction, weather context, and music synthesis.
AI- ChatGPT, Claude were used in the creation of this model and dataset
