This is a fully merged Transformers checkpoint of unsloth/Qwen3.5-4B, fine-tuned to extract structured Cityscapes road-scene facts from an image. It predicts:

presence for 2 surface classes and 11 object classes;
a 3×3 image zone set for each present class;
count buckets for the 11 object classes;
conservative subject on road|sidewalk contact relations.

Both the language model and vision tower LoRA weights are merged into this checkpoint. It is not an adapter repository and does not require PEFT or the original LoRA at inference time.

The model is specialized for this fixed Cityscapes JSON task. It is not a general object detector and the evaluation below does not imply a general vision-language improvement.

Files

The model is stored as two safe-serialized shards:

File	Size	SHA-256
`model.safetensors-00001-of-00002.safetensors`	5,329,398,688 bytes	`0acfb9ce10a5057b9f7ef85a83d41e3999cb608032594d7e9e5a557c8cdc4b03`
`model.safetensors-00002-of-00002.safetensors`	3,990,429,408 bytes	`b521e3cf44161c13e23878c5b416fb1f24613d3428f2981bd440a94dc7568c1b`

There are 738 model tensors: 690 BF16 and 48 F32. The F32 tensors are the base architecture's numerically sensitive parameters, not unmerged LoRA weights. See release_manifest.json for machine-readable provenance, checksums, package versions, and merge validation.

Quick start with Transformers

Use an explicit schema prompt. The input should place the image before the question, matching training and evaluation.

python

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "Singularity87/Cityscapes-Qwen3.5-4B"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

image = Image.open("frankfurt_000000_000294_leftImg8bit.png").convert("RGB")

system_prompt = """You extract structured Cityscapes facts from an image.
Return raw JSON only with exactly the top-level keys surfaces, objects, relations.
surfaces must contain road and sidewalk with present and zones.
objects must contain person, rider, car, truck, bus, train, motorcycle,
bicycle, traffic_light, traffic_sign, and pole with present, count, and zones.
Valid zones are upper_left, upper_center, upper_right, middle_left,
middle_center, middle_right, lower_left, lower_center, lower_right.
Valid counts are 0, 1, 2-3, 4-7, 8+, unknown. An absent object is
{"present":false,"count":"0","zones":[]}. Relations use subject, relation,
object; relation is on and object is road or sidewalk."""

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {
                "type": "text",
                "text": (
                    "Analyze this urban road image and return one completed "
                    "JSON object in the required schema."
                ),
            },
        ],
    },
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {key: value.to(model.device) for key, value in inputs.items()}

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
    )

new_tokens = generated[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0])

What was done

Data

Cityscapes fine annotations and left images were converted into an image-first supervised JSON task.
Training split: 2,975 images.
Validation split: 500 images.
Cityscapes images and annotations are not redistributed in this model repository.

The output contract uses:

surfaces: road, sidewalk;
objects: person, rider, car, truck, bus, train, motorcycle, bicycle, traffic_light, traffic_sign, pole;
zones: a 3×3 grid from upper_left through lower_right;
count buckets: "0", "1", "2-3", "4-7", "8+", "unknown";
relations: one of the eight instance classes, relation "on", and surface road or sidewalk.

Training

This was 16-bit LoRA SFT, not QLoRA. The base model was loaded in BF16 with neither 4-bit nor 8-bit base quantization. Loss was calculated on assistant tokens only. Images retained Cityscapes resolution through the Unsloth resize="max" vision collator behavior.

Setting	Value
Base revision	`3764fa359b9082ea5a1e4a5e3ac3aaf6e9671636`
Epochs / optimizer steps	1 / 372
Train / eval batch size	8 / 4
Gradient accumulation	1
Precision	BF16, TF32 enabled
LoRA rank / alpha / dropout	16 / 16 / 0
LoRA scope	language + vision; attention + MLP
Learning rate	`1e-4`
Scheduler / warmup	cosine / 5%
Optimizer / weight decay	AdamW Torch / `0.001`
Maximum sequence length	4,096
Seed	3,407
Hardware	NVIDIA RTX 6000 Ada Generation, 48 GB
Train runtime	5,018.83 seconds

The training command in the source project was equivalent to:

bash

python -m scripts.training.train_qwen35_cityscapes_lora \
  --model-name unsloth/Qwen3.5-4B \
  --epochs 1 \
  --batch-size 8 \
  --eval-batch-size 4 \
  --gradient-accumulation-steps 1 \
  --learning-rate 1e-4 \
  --max-length 4096 \
  --lora-rank 16 \
  --lora-alpha 16 \
  --lora-dropout 0 \
  --assistant-only-loss \
  --finetune-vision-layers \
  --finetune-language-layers \
  --finetune-attention-modules \
  --finetune-mlp-modules \
  --seed 3407

Full language-and-vision merge

The adapter contained 688 tensors: 496 language tensors and 192 vision tensors. A llama.cpp runtime --lora would not apply the vision part, so the complete adapter was merged first:

python

import torch
from unsloth import FastVisionModel

model, processor = FastVisionModel.from_pretrained(
    model_name="final_adapter",
    max_seq_length=4096,
    dtype=torch.bfloat16,
    load_in_4bit=False,
    load_in_8bit=False,
    load_in_16bit=True,
    use_gradient_checkpointing=False,
)
model.save_pretrained_merged(
    "Cityscapes-Qwen3.5-4B",
    processor,
    save_method="merged_16bit",
    safe_serialization=True,
    max_shard_size="4GB",
)

Header validation found 738 merged tensors: 441 language and 297 vision, with zero tensor names containing lora_ and no adapter_config.json.

How it was evaluated

The formal comparison used the BF16 GGUF export of this same merged checkpoint against the untouched BF16 base model under identical llama.cpp settings:

all 500 Cityscapes validation images;
OpenAI-compatible /v1/chat/completions;
the same explicit schema system prompt and one text-only formatting example for both models;
image-first request;
temperature=0, seed=3407, max_tokens=1024;
thinking disabled;
no JSON grammar and no response_format;
10,000 paired bootstrap samples with seed 3,407.

task_score is the equal-weight mean of presence macro F1, per-class count accuracy, zone micro F1, and relation micro F1.

Results

Strict all-sample scoring:

Metric	Base BF16	Tuned BF16	Delta
JSON valid rate	0.292	1.000	+0.708
Strict schema valid rate	0.214	1.000	+0.786
Task score	0.1840	0.8054	+0.6214

To separate formatting gains from semantic gains, a diagnostic removed only a single whole-response Markdown JSON fence and then selected the 402/500 rows where both outputs passed the same strict schema:

Metric	Base BF16	Tuned BF16	Delta	Paired-bootstrap 95% CI
Presence macro F1	0.8110	0.8989	+0.0879	[0.0619, 0.1156]
Count macro accuracy	0.7280	0.8211	+0.0932	[0.0821, 0.1043]
Zone micro F1	0.3495	0.8447	+0.4953	[0.4857, 0.5046]
Relation micro F1	0.5711	0.6513	+0.0802	[0.0606, 0.1001]
Task score	0.6149	0.8040	+0.1891	[0.1790, 0.2000]

On the stricter unnormalized joint-valid subset (107/500), task score improved from 0.6434 to 0.8134; the delta 95% CI was [0.1494, 0.1918]. The predefined pass gate was satisfied. The largest semantic gain was 3×3 zone localization.

Validation loss at the end of training was 0.0446758; this was not used by itself as evidence that the tuned model beat the base model.

Reproducing the paired evaluation

With a base BF16 server on port 8080 and this tuned BF16 model on port 8081:

bash

python -m scripts.evaluation.evaluate_cityscapes_llamacpp \
  --base-url http://127.0.0.1:8080 \
  --tuned-url http://127.0.0.1:8081 \
  --eval-file data/cityscapes-agent-sft/cityscapes_agent_sft_val.jsonl \
  --prompt-profile explicit-schema \
  --seed 3407 \
  --bootstrap-samples 10000

The exact evaluation prompt SHA-256 was b7b80a3b18578acb8406af229392f9d5585563ecbace3aac724145f2d8187a3f.

Limitations

Results apply only to the fixed Cityscapes structured JSON task.
Counts are buckets, not exact detections.
Relations intentionally cover only direct on road|sidewalk contact.
The model can still hallucinate or miss small/occluded objects.
Use of this checkpoint is restricted to non-commercial purposes, and citing the Cityscapes Dataset is a condition of that use. See License and Citation.

Software

PyTorch 2.10.0
Transformers 5.5.0
PEFT 0.19.1
Safetensors 0.8.0
TRL 0.24.0
Unsloth 2026.7.2

License

The effective terms for this checkpoint are the intersection of two licenses, and that intersection is non-commercial.

The base model unsloth/Qwen3.5-4B is Apache-2.0.
The Cityscapes Dataset License also applies, because these fine-tuned weights are a derivative work of the dataset. It states that you may not use the dataset or any derivative work for commercial purposes, such as licensing or selling the data, or using the data with a purpose to procure a commercial gain.

This repository is therefore not labelled Apache-2.0. Apache-2.0 on its own would grant commercial rights that the Cityscapes terms withhold. If you need commercial use, take that up with the Cityscapes authors rather than relying on the base model's license.

The Cityscapes license does permit distributing abstract derivative works such as trained models, provided they do not allow the dataset to be recovered, which is what makes publishing this checkpoint possible. It does not permit redistributing the dataset itself, so no Cityscapes images or annotations are included here.

Citation

Referencing the Cityscapes Dataset in any work that uses this model is a condition of the dataset license, not a courtesy.

bibtex

@inproceedings{Cordts2016Cityscapes,
  title     = {The Cityscapes Dataset for Semantic Urban Scene Understanding},
  author    = {Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and
               Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and
               Franke, Uwe and Roth, Stefan and Schiele, Bernt},
  booktitle = {Proc. of the IEEE Conference on Computer Vision and Pattern
               Recognition (CVPR)},
  year      = {2016}
}