Repository: icewaterdun/Qwen2.5-72B-Instruct-PyQGIS Base model: Qwen2.5-72B-Instruct Adapter type: LoRA (PEFT) Checkpoint used: checkpoint-600 (early-stop selection)

Model Summary

This repository contains a LoRA adapter that tailors Qwen2.5-72B-Instruct to PyQGIS/QGIS workflows (e.g., geoprocessing, vector/raster operations, layer styling, and common Geo/OGR tasks). The adapter focuses on instruction-following in a GIS setting, emphasizing practical coding patterns and on-tool terminology.

Why LoRA? Lower VRAM footprint, faster iteration, and easy plug-and-play with the base model.
Best checkpoint: checkpoint-600. Later checkpoints (800+) showed a slight regression (on the same eval split), suggesting mild overfitting.
Training config: See the uploaded YAML in this repo for exact hyperparameters and data settings.

Evaluation

Evaluated on the same held-out set (503 items). We report cross-entropy loss and the corresponding perplexity (lower is better). Perplexity is computed from the reported loss via: [ \text{PPL} = \exp(\text{loss}) ] assuming the loss is the mean token-level cross-entropy in nats (Hugging Face Transformers’ default).

Model / Checkpoint	eval_loss	Perplexity (PPL)
Baseline (no LoRA)	2.476	11.894
ckpt-400	0.885	2.423
ckpt-600 (selected)	0.865	2.375
ckpt-800	0.898	2.455

Takeaway: The LoRA adapter substantially improves the base model on this PyQGIS-focused eval set (PPL ↓ from ~11.9 → ~2.38). Among LoRA checkpoints, 600 is best (lowest loss/PPL), while 800 begins to drift upward.

How perplexity is computed (quick note)

If your framework reports cross-entropy loss in nats (the usual case in 🤗 Transformers), PPL = exp(loss).
If the loss were measured in bits instead, it would be PPL = 2^(loss).
The reported PPL is the exponentiated average token-level loss over the evaluation set.

Data & Training

Data domain: PyQGIS/QGIS instructions and code-centric prompts (see dataset_dir and datasets referenced in the training YAML).
Template: qwen chat template.
Sequence lengths: cutoff length was 1024 tokens for both prompt and response packing in eval/training runs shown here.
Precision: bf16 for evaluation; training precision and optimizer settings are in the YAML.
Adapter: LoRA; rank/alpha/targets as specified in the uploaded training YAML.

For exact hyperparameters (optimizer, scheduler, LoRA rank/alpha/dropout, target modules, warmup, batch sizes, etc.), consult the training YAML included in this repository.

Intended Use

Assisting with PyQGIS scripting and QGIS-related problem solving.
Generating and explaining PyQGIS code snippets for common GIS tasks.

Limitations

The adapter focuses on PyQGIS-specific tasks; general world knowledge or unrelated domains remain the responsibility of the base model.
Hallucinations are still possible—especially for niche plugins, exotic projections, or very new QGIS APIs.
Not evaluated for safety-critical, legal, or medical use.

License

Inherits the license and usage terms of Qwen2.5-72B-Instruct for inference.
The LoRA weights here are released for research and non-commercial evaluation; please review the base model’s license before any production use.

Citation

If you use this adapter, please cite the base model and this repository.

text

@misc{Qwen2.5-72B-Instruct-PyQGIS,
  title  = {Qwen2.5-72B-Instruct-PyQGIS (LoRA)},
  author = {icewaterdun},
  year   = {2025},
  note   = {LoRA adapter for PyQGIS tasks on top of Qwen2.5-72B-Instruct}
}