Usage and License Notices: This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses, including but not limited to the OpenAI Terms of Use for the dataset and the specific licenses for base language models for checkpoints trained using the dataset (e.g. Llama community license for LLaMA-2 and Vicuna-v1.5). This project does not impose any additional constraints beyond those stipulated in the original licenses. Furthermore, users are reminded to ensure that their use of the dataset and checkpoints is in compliance with all applicable laws and regulations.

The folder of table_01 contains all scripts to generate table 1 in the paper.

Quantitative Reasnoning Test

Qualitative Reasnoning Test

The folder of Figures contains all scripts to generate figures 5-9 in the paper.

With respect to figures 5 and 6, the generated figures with .png are saved in the same directory.(Note: Back to relative home directory!)

With respect to figures 7, 8, and 9. The generated results are stored in vqa_inference_results_789.jsonl. The answers appear in the text field. This file is a text-based format and can be opened with Notepad. (Note: Back to relative home directory! )

We provide the training scripts, including citysense-gpt-finetune.sh and citysense-gpt-lora-merge.sh. It should be noted that the training configurations are compatible with B200 GPUs.

Training data set is shared through the link : https://drive.google.com/file/d/1D8zjGqoFON1zrgeKhsQDvdVcBXgmAPS6/view?usp=sharing

Go to the folder of inference. NOTE: The code was tested on L4 GPUs. Using higher-end GPUs may require additional configuration and could lead to compatibility issues.

If you encounter OSError: [WinError 1314] when downloading the 14.1GB dataset or model checkpoints, it is a Windows-specific permission issue related to symbolic links.