Naija-Petro 8B is a Qwen3-8B model fine-tuned (QLoRA, Unsloth) on ~20,000 synthetic petroleum-engineering instruction-response pairs. It is the lightweight, fast-inference variant of the Naija-Petro family and the model served behind the project's retrieval-augmented assistant.
⚠️ The base training data is general/global petroleum knowledge. For Nigeria-specific facts (regulation, the PIA 2021, NUPRC/NMDPRA/NNPC), pair this model with the Naija-Petro RAG system, which grounds answers in verifiable Nigerian sources.
Model details
Developed by: Naija-Petro project (Hugging Face: Shinzmann)
Model type: Decoder-only causal LM, instruction-tuned
Technical question answering and explanation across petroleum-engineering subdomains: concepts, equations and derivations, workflow guidance, and terminology, as a study aid and engineering decision-support tool.
Downstream use
Backbone for retrieval-augmented assistants (see the project repo), further domain fine-tuning, or distillation.
Out-of-scope use
Not for autonomous operational, safety-critical, or financial decisions; not a substitute for licensed engineering judgment, official regulations, or field data. General-purpose chat is not its focus.
Bias, risks, and limitations
Trained largely on synthetic data generated from a scraped corpus; it can be confidently wrong ("hallucinate"), especially on numerical specifics and Nigeria-specific regulation/economics.
English only. May reflect biases of its base model and source literature.
Knowledge is static as of training; use the RAG layer for current/local facts.
Recommendation: Always validate outputs with qualified engineers and primary sources before any operational use.
How to get started
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model_id ="Shinzmann/naija-petro-8b"4tok = AutoTokenizer.from_pretrained(model_id)5model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")67SYSTEM =(8"You are Naija-Petro, an expert petroleum-engineering AI assistant. Provide "9"precise, technically accurate answers; include equations, units, and "10"practical considerations."11)12messages =[13{"role":"system","content": SYSTEM},14{"role":"user","content":"Explain the material balance equation for an undersaturated reservoir."},15]16inputs = tok.apply_chat_template(messages, add_generation_prompt=True,17 enable_thinking=False, return_tensors="pt").to(model.device)18out = model.generate(inputs, max_new_tokens=512, temperature=0.4, top_p=0.9)19print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
Ollama (GGUF):
ollama run hf.co/Shinzmann/naija-petro-8b-GGUF:Q4_K_M
Training details
Data
~20,000 instruction-response pairs generated with NVIDIA NeMo Data Designer from a scraped, de-duplicated petroleum corpus (arXiv, Semantic Scholar, OpenAlex, Crossref, DOE/OSTI, PetroWiki, the SLB glossary, EIA, and more), with an LLM-as-judge quality-scoring pass. Pipeline and EDA are in the project notebooks.
Procedure
QLoRA (4-bit NF4) with Unsloth on a single A100 80GB.
Hyperparameter
Value
Base model
Qwen3-8B
Method
QLoRA, 4-bit NF4
LoRA rank / alpha / dropout
32 / 64 / 0.0
Epochs
3
Effective batch size
64 (32 × 2 grad-accum)
Learning rate / scheduler
2e-4 / cosine, 5% warmup
Weight decay
0.01
Max sequence length
2048
Seed
42
Training results (Weights & Biases)
Trained for 3 epochs (669 optimiser steps) on a single A100 80GB.
Metric
Value
Final training loss
≈ 0.83
Final validation loss
≈ 0.86
Epochs / steps
3 / 669
Evaluation
Training converged to a low validation loss (≈ 0.86; see Training results). A downstream 30-question internal benchmark across six subdomains (drilling, reservoir, production, completions, EOR, well testing), scored by an LLM-as-judge on technical accuracy, completeness, and terminology, was also run; the automated judge did not differentiate reliably in that pass, so a corrected evaluation is in progress and task-level scores are not yet reported. Qualitatively, the fine-tuned model produces well-structured, equation- and unit-aware answers in the target domain. Treat all outputs as expert-validated decision support.
