YOLOv8 Urban Segmentation (Vegetation & Sky)
1. Model Description & Intended Use
This repository contains a set of YOLO models trained for instance segmentation, specifically focused on the urban environment at the pedestrian level. The central objective is to automate the identification and delimitation of two exact categories to calculate fundamental urban metrics:
- Vegetation (
vegetation): Used for measuring the Green View Index (GVI) on sidewalks.
- Sky (
sky): Used for measuring the Sky View Factor (SVF).
The models are designed to operate optimally on street-level images (Street View), primarily capturing sidewalks, intersections, and parks.
2. Data Methodology
To build the dataset, the alivestreets tool was used, enabling the automated download of Google Street View (GSV) images.
- Dataset Volume: A standardized dataset of 1,000 strictly selected images was consolidated.
- Annotation: Labeling was performed on the Roboflow platform using the SAM 3 model for polygon instance segmentation, followed by manual refinement.
- Data Split: The images were divided into 80% for Training, 10% for Validation, and 10% for Testing.
3. Available Models
As part of the scalability strategy, three distinct models were trained sequentially following the same data protocol. The following weights (.pt) are available in this repository:
mexicali_yolov8m.pt: Trained exclusively with the curated dataset from Mexicali.cdmx_yolov8m.pt: Trained exclusively with the curated dataset from Mexico City (CDMX).combined_yolov8s.pt: Trained by merging both datasets to evaluate the algorithm's generalization capabilities across different geographical regions. A lighter architecture (YOLOv8 small) was chosen for this model as it acted as a natural regularizer, forcing the network to learn fundamental urban features rather than memorizing parameters.
4. Results & Metrics (Test Set)
The following metrics reflect the real performance of the models on the Test set (images never seen before by the algorithm).
| Model | Architecture | mAP@50 | Precision | Recall | F1-Score |
|---|
| Mexicali | YOLOv8m | 73.3% | 79.0% | 77.0% | 77.9% |
| CDMX | YOLOv8m | 84.1% | 79.2% | 83.6% | 81.1% |
| Combined | YOLOv8s | 79.3% | 81.7% | 76.5% | 78.9% |
| Combined | YOLOv8m | 66.4% | 83.6% | 59.4% | 68.9% |
5. Error Analysis & Limitations
Inference analysis revealed asymmetric performance between the detected classes. While the vegetation category showed stable behavior, the sky category presented a higher failure rate, affecting the overall mAP50.
This is primarily caused by an intrinsic class imbalance in Street View images; the pixel proportion and representation of vegetation significantly outvolume the clear sky areas visible between buildings. Due to limitations in the web training platforms used, it was not possible to apply an asymmetric penalty (modifying the cls_pw or class positive weight hyperparameter) to force the model to pay more attention to the minority class.
6. Future Work
To bring the GVI and SVF calculations to a production level, the following steps are planned:
- Local Training Environment: Development of a local Python training script to eliminate reliance on closed platforms. This will allow tuning advanced hyperparameters like
cls_pw to mitigate the "sky" category imbalance.
- Migration to YOLO26: The inference ecosystem will be migrated to YOLO26 to ensure long-term support. Preliminary experiments with this native NMS-free version show that segmentation metrics remain stable, ensuring a deterministic and efficient model.
