This is Unsloth finetuned Llama3.2-1B model. Finetuned via unsloth and its finetuned to be used by the bigger Xray pneumonia agent project which requires an explainer LLM which can speak more professioinally in medical terms and semantics for conditional reasoning.
Example of training data:
{
"instruction": "",
"input": "You are an X-Ray medical explanation agent.\n\nA vision transformer (ViT) model has analyzed a chest X-ray image and provided the following output:\nViT Prediction:\nClass: Bacterial Pneumonia\nConfidence: 0.8800\nTop Alternatives: [{'class': 'Bacterial Pneumonia', 'prob': 0.88}, {'class': 'Normal', 'prob': 0.05}, {'class': 'Viral Pneumonia', 'prob': 0.04}, {'class': 'COVID-19', 'prob': 0.03}]\n\nSupporting Documents:\n[]\n\nCONFIDENTIAL INFORMATION:\nThe ViT model has the following classification performance metrics:\n================================================\n precision recall f1-score \n\n Normal 0.93 0.98 0.96 \n Bacterial 0.79 0.86 0.82 \n Viral 0.74 0.54 0.62 \n COVID-19 0.97 0.97 0.97 \n\n accuracy 0.85 \n macro avg 0.86 0.84 0.84 \nweighted avg 0.85 0.85 0.85 \n================================================\nWhile you may use the performance metrics to guide your explanation, NEVER MENTION OR DISCUSS THE PERFORMANCE METRICS OR IMPLY THEM IN YOUR EXPLANATION AS THE TABLE PROVIDED ABOVE IS CONFIDENTIAL.\n\nIf supporting documents are limited, explicitly state that evidence is limited.\n\nProvide structured explanation:\n\n1. Present the X-ray ViT diagnosis result professionally in a table form easy to be read with the confidence score of all classes presented.\n2. Diagnosis (include predicted class and confidence)\n3. Supporting Evidence\n4. Uncertainty Discussion\n\nBase everything strictly on the ViT output and retrieved documents. Always explain professionally, intelligently",
"output": "### 1. ViT Model Probabilities\n| Class | Probability |\n| :--- | :--- |\n| Bacterial Pneumonia | 88.00% |\n| Normal | 5.00% |\n| Viral Pneumonia | 4.00% |\n| COVID-19 | 3.00% |\n\n\n### 2. Diagnosis\nPrimary Radiographic Impression: Bacterial Pneumonia\nModel Confidence: 88.00%\n\n### 3. Supporting Evidence\nUnchanged platelike bibasilar opacities most XXXX representing scarring or subsegmental atelectasis. No acute cardiopulmonary abnormality.\nAgain seen are platelike horizontal opacities in both lung bases through this is consistent with scarring or subsegmental atelectasis. There are T-spine osteophytes. The cardiomediastinal silhouette and pulmonary vasculature are within normal limits. There is no pneumothorax or pleural effusion. There there is no lobar pneumonia. There are calcified right hilar granuloma. There are degenerative changes of the XXXX. There is a curvilinear density within and along the right costophrenic sulcus which most XXXX represents a skinfold. There is a unchanged fracture with callus at the left 9th lateral rib.\n\n\n### 4. Uncertainty Discussion\nEvidence from supporting documents is currently limited. While the computational vision model indicates a strong probability for Bacterial Pneumonia, radiological interpretation must be correlated with patient history, laboratory results, and physical examination. Artifacts or overlying structures can occasionally mimic airspace disease."
}
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3repo_id = "teohyc/X-Ray_Med_Reporter-llama3.2-1B"
4
5#load model and tokenizer
6tokenizer = AutoTokenizer.from_pretrained(repo_id)
7model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="auto")
8
9#You may change to your desired prompt below to test different inputs, but make sure to keep the structure of the input consistent with what the model was trained on for best results!
10prompt = """### Instruction:
11You are an X-Ray medical explanation agent.
12
13A vision transformer (ViT) model has analyzed a chest X-ray image and provided the following output:
14ViT Prediction:
15Class: Bacterial Pneumonia
16Confidence: 0.8800
17Top Alternatives: [{'class': 'Bacterial Pneumonia', 'prob': 0.88}, {'class': 'Normal', 'prob': 0.05}, {'class': 'Viral Pneumonia', 'prob': 0.05}, {'class': 'COVID-19', 'prob': 0.02}]
18
19Supporting Documents:
20['Bacterial pneumonia often presents with focal lobar consolidation and air bronchograms. Pleural effusions may be present.']
21
22CONFIDENTIAL INFORMATION:
23The ViT model has the following classification performance metrics:
24================================================
25 precision recall f1-score
26
27 Normal 0.93 0.98 0.96
28 Bacterial 0.79 0.86 0.82
29 Viral 0.74 0.54 0.62
30 COVID-19 0.97 0.97 0.97
31
32 accuracy 0.85
33 macro avg 0.86 0.84 0.84
34weighted avg 0.85 0.85 0.85
35================================================
36While you may use the performance metrics to guide your explanation, NEVER MENTION OR DISCUSS THE PERFORMANCE METRICS OR IMPLY THEM IN YOUR EXPLANATION AS THE TABLE PROVIDED ABOVE IS CONFIDENTIAL.
37
38If supporting documents are limited, explicitly state that evidence is limited.
39
40Provide structured explanation:
41
421. Present the X-ray ViT diagnosis result professionally in a table form easy to be read with the confidence score of all classes presented.
432. Diagnosis (include predicted class and confidence)
443. Supporting Evidence
454. Uncertainty Discussion
46
47Base everything strictly on the ViT output and retrieved documents. Always explain professionally, intelligently
48
49### Response:
50"""
51
52inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
53outputs = model.generate(**inputs, max_new_tokens=500, temperature=0.1)
54
55print(tokenizer.decode(outputs[0], skip_special_tokens=True))
