This model is part of the GeoLLM-Qwen3.5-FineTune benchmark. All five model sizes (0.8B--27B) were trained with identical hyperparameters on the same dataset for fair comparison.

Performance

Metric	Base	Fine-tuned
Overall weighted score	0.341	0.353
QA ROUGE-L	0.1297	0.1931
QA BERTScore	0.8143	0.8533
CoT ROUGE-L	0.1270	0.1953
Hallucination pass rate	66.7%	33.3%
Training loss	--	1.316

See the full benchmark comparison for all models.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AshkanTaghipour/GeoLLM-Qwen3.5-4B", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("AshkanTaghipour/GeoLLM-Qwen3.5-4B")

messages = [
    {"role": "system", "content": "You are a specialist geologist with expertise in Western Australian mineral exploration."},
    {"role": "user", "content": "What geophysical methods would you recommend for targeting komatiite-hosted nickel sulphide deposits in the Eastern Goldfields?"},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Training Details

Method: bf16 LoRA (r=16, alpha=16) via Unsloth + SFTTrainer
Epochs: 5
Optimizer: adamw_8bit, cosine LR schedule (2e-4, 10% warmup)
Dataset: mineral-exploration-geology-qa (479 train, 26 test examples)
Hardware: NVIDIA A100-80GB
VRAM (training): 10 GB
VRAM (inference): ~9 GB

Author

Ashkan Taghipour -- GitHub | HuggingFace