GlobalHealthAtlas Public Evaluator
The GlobalHealthAtlas Public Evaluator is a fine-tuned version of Qwen3-8B (via LoRA) designed to assess Large Language Model (LLM) outputs along six critical dimensions of public health reasoning: Accuracy, Reasoning, Completeness, Consensus Alignment, Terminology Norms, and Insightfulness.
Model Description
This model acts as a domain-aligned evaluator distilled from high-confidence judgments of diverse LLMs. It is intended to be used as a ranking/scoring component for the GlobalHealthAtlas project, enabling reproducible evaluation of LLMs for safety-critical public health reasoning.
Intended Uses & Limitations
This model is intended to be used as a scoring component (scorer) for public health tasks, such as:
- Relevancy scoring of health documents.
- Automated quality assessment of reasoning chains.
- Content prioritization in public health contexts.
Disclaimer: This model is NOT intended for clinical diagnosis, medical advice, or other high-stakes decision-making. Users should validate performance on their own data prior to deployment.
Usage
To use the GlobalHealthAtlas Public Evaluator, clone the official repository and follow the instructions in the codebase.
Example: Running the Scorer
1cd scoring
2python scorer.py --input-file ../data/input.json --output-file ../data/output.json
Batch Scoring
1cd "Source Code/Public Evaluator"
2python scorer_batch.py
Training Procedure
Training Hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- optimizer: AdamW
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 2.0
Framework Versions
- PEFT 0.15.1
- Transformers 4.51.3
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0
Citation
If you use this model or the GlobalHealthAtlas dataset, please cite:
1@article{globalhealthatlas2026,
2 title={From Knowledge to Inference: Scaling Laws of Specialized Reasoning on GlobalHealthAtlas},
3 author={GlobalHealthAtlas Team},
4 year={2026}
5}
