This paper presents a novel approach to Geo-Localization, a task
that aims to predict geographic coordinates, i.e., latitude and
longitude of an image based on its visual content. Traditional
methods in this domain often rely on databases,
complex pipelines or large-scale image classification networks.
In contrast, we propose a direct regression approach that
simplifies the process by predicting the geographic coordinates
directly from the image features. We leverage a pre-trained
Vision Transformer (ViT) model, specifically a pre-trained CLIP
model, for feature extraction and introduce a regression head
for coordinate prediction. Various configurations, including pre-
training and task-specific adaptations, are tested and evaluated
resulting in our model called ReGeo. Experimental results show
that ReGeo offers competitive performance compared to existing
SOTA approaches, despite being simpler and needing minimal
supporting code pipelines.
1# imports
2import torch
3from PIL import Image
4from model import LocationDecoder # ReGeo model class: https://github.com/TobiasRothlin/GeoLocalization/blob/main/src/DGX1/src/RegressionPretraining/Model.py
5from transformers import CLIPProcessor
6
7# load custom config (do not use AutoConfig), an example can be found in this repo
8config = { ... }
9
10device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
11preprocessor = CLIPProcessor.from_pretrained('openai/clip-vit-large-patch14-336')
12model = LocationDecoder.from_pretrained('OSTswiss/ReGeo', config=config)
13
14# Load model for inference
15model.to(device)
16model.eval()
17
18# load image
19image_path = 'path/to/your/image.jpg' # can be any size
20image = Image.open(image_path)
21model_input = preprocessor(images=image, return_tensors="pt")
22pixel_values = model_input['pixel_values'].to(device)
23
24# run inference
25with torch.no_grad():
26 output = model(pixel_values)
27 normal_coordinates = output.squeeze().tolist()
28 latitude = normal_coordinates[0] * 90
29 longitude = normal_coordinates[1] * 180
