Mistral 7B Instruct v0.3 fine-tuned with QLoRA for medical question answering in Portuguese and English, with additional support for Brazilian healthcare protocols.

Developed as part of the Tech Challenge Phase 3 — Post-Graduate Program in AI for Developers at FIAP (São Paulo, Brazil).

Overview

This repository contains LoRA adapters (not the full model) trained on the MedQuAD dataset translated to Portuguese, combined with Brazilian-specific medical QA pairs covering diseases prevalent in Brazil and clinical guidelines from the Brazilian public health system (SUS — Sistema Único de Saúde).

The model is designed to answer medical questions clearly and accurately, integrated into a multi-agent system with RAG and safety guardrails.

WARNING: This is an academic project. It must NOT be used for real clinical decisions. All responses must be validated by a qualified healthcare professional.

Developed by: Adriana Martins, Diego Oliveira, Eduardo Zagari, Renan Torres
Base model: mistralai/Mistral-7B-Instruct-v0.3
Type: LoRA Adapter (PEFT) for Causal LM
Languages: Portuguese (80%), English (20%)
License: Apache 2.0
Repository: github.com/zagari/virtual-medical-assistant

Model Selection: Checkpoint 1500

This model corresponds to checkpoint-1500 from a 2-epoch training run (~1.3 epochs completed). Extensive evaluation across 7 checkpoints revealed that this checkpoint achieves the best balance between learning and avoiding catastrophic collapse:

Metric vs Ground Truth	Baseline	This Model	Improvement
BLEU	0.039	0.298	+664%
ROUGE-1 F1	0.318	0.517	+63%
ROUGE-L F1	0.175	0.421	+141%
Semantic Similarity	0.904	0.939	+3.9%

How to Use

With Transformers + PEFT

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zagari/medical-assistant-mistral-7b-ft"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="float16",
    device_map="auto",
)

With Unsloth (faster)

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="zagari/medical-assistant-mistral-7b-ft",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

Prompt Format (Alpaca Template)

Abaixo está uma instrução que descreve uma tarefa, junto com uma entrada que
fornece contexto adicional. Escreva uma resposta que complete adequadamente
a solicitação.

### Instrução:
Responda a seguinte pergunta médica de forma clara e detalhada.

### Entrada:
Quais são os sintomas do diabetes tipo 2?

### Resposta:

Training Data

Base dataset: MedQuAD — Medical Question Answering Dataset (NIH sources: NCI, NIDDK, NINDS, NHLBI, GHR, MedlinePlus)
Total size: ~20,500 question-answer pairs
Composition:
- 80% Portuguese: MedQuAD translated to Brazilian Portuguese
- 20% English: Original MedQuAD entries retained for technical terminology coverage
- Brazilian supplement: QA pairs derived from Brazilian clinical guidelines (Linhas de Cuidado from the Ministry of Health / SUS), covering conditions with high prevalence in Brazil such as dengue, Chagas disease, sickle cell anemia, tuberculosis, and tropical infectious diseases
Format: Alpaca JSONL (instruction / input / output)

Training Details

Hyperparameters

Parameter	Value
Method	QLoRA 4-bit
LoRA rank (r)	64
LoRA alpha	128
LoRA dropout	0.05
Target modules	q, k, v, o, gate, up, down
Trainable parameters	167M (4.27% of total)
Epochs	2 (stopped at ~1.3 epochs / 1500 steps)
Effective batch size	16
Learning rate	2e-4
Scheduler	Cosine (decayed to ~0 at step 1500)
Warmup steps	200
Optimizer	AdamW 8-bit
Max sequence length	2,048 tokens
Precision	FP16

Training Results

Metric	Value
Total steps	1,500
Final training loss	0.576
Best eval loss	0.654 (step 1000)
Training time	~13 hours

Why Checkpoint 1500?

Multi-checkpoint evaluation revealed catastrophic collapse between steps 1,300-1,500 in longer training runs (5 epochs). The cosine LR schedule with max_steps=1500 decays the learning rate to near-zero by step 1,300 (LR ≈ 6.6e-6), preventing the collapse observed in runs with higher LR at that point.

Infrastructure

GPU: NVIDIA RTX A5000 (24GB VRAM)
Platform: Linux
Libraries: Unsloth 2026.3.x, Transformers 5.2.0, PEFT 0.18.1, PyTorch 2.10.0

Intended Use

This model serves as the reasoning component of a multi-agent virtual medical assistant built with LangGraph:

Triage Agent: classifies the physician's query
Protocol Agent: semantic search (RAG) over clinical guidelines via ChromaDB
Patient Data Agent: queries patient records from PostgreSQL
Reasoning Agent: synthesizes a clinical response using this fine-tuned model
Guardrails Agent: validates safety and compliance of the response
Explainability Agent: attaches sources and confidence level

GGUF Version

A quantized GGUF version (Q4_K_M, ~4GB) is available for CPU/Apple Silicon inference:

Repository: zagari/medical-assistant-mistral-7b-ft-gguf

Limitations

Academic project — not clinically validated
Coverage: limited to conditions present in MedQuAD and Brazilian supplements (primarily common diseases)
Language: Portuguese responses may occasionally contain calques from English source material
Never prescribes medication — by design, all responses are suggestions that require validation by a healthcare professional

Citation

bibtex

@misc{medical-assistant-fiap-2026,
  title={Virtual Medical Assistant with Fine-Tuning and LangGraph},
  author={Martins, Adriana and Oliveira, Diego and Zagari, Eduardo and Torres, Renan},
  year={2026},
  note={Tech Challenge Phase 3 — FIAP Post-Graduate Program in AI for Developers}
}