Views
No views yet
This is a LoRA adapter (~1.1 GB) — it must be loaded on top of the merged Stage 2 modelE27085921/HIKARI-Sirius-8B-SkinDx-RAG.⚠️ Important: Do NOT load this on top of rawQwen/Qwen3-VL-8B-Thinking. The merged-init design requires disease knowledge to already be in the base weights. UseHIKARI-Sirius-8B-SkinDx-RAG(merged) as the base.✅ Advantage: Lightweight — download only ~1.1 GB instead of ~17 GB (plus the ~17 GB Stage 2 base).💾 If you prefer a standalone ready-to-use model, see the merged version: E27085921/HIKARI-Vega-8B-SkinCaption-Fused (~17 GB)
1from peft import PeftModel
2from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
3import torch
4from PIL import Image
5
6# Step 1: Load the MERGED Stage 2 disease model as base
7# (disease knowledge is permanently in its weights — DO NOT use raw Qwen base here)
8base = Qwen3VLForConditionalGeneration.from_pretrained(
9 "E27085921/HIKARI-Sirius-8B-SkinDx-RAG", # merged Stage 2 weights (~17 GB)
10 torch_dtype=torch.bfloat16,
11 device_map="auto",
12 trust_remote_code=True,
13)
14
15# Step 2: Apply Stage 3 caption LoRA adapter (~1.1 GB)
16model = PeftModel.from_pretrained(base, "E27085921/HIKARI-Vega-8B-SkinCaption-Fused-LoRA")
17processor = AutoProcessor.from_pretrained("E27085921/HIKARI-Vega-8B-SkinCaption-Fused-LoRA", trust_remote_code=True)
18
19# Step 3: Inference — see full examples at E27085921/HIKARI-Vega-8B-SkinCaption-Fused
20image = Image.open("skin_lesion.jpg").convert("RGB")1@misc{hikari2026,
2 title = {HIKARI: RAG-in-Training for Skin Disease Diagnosis
3 with Cascaded Vision-Language Models},
4 author = {Watin Promfiy and Pawitra Boonprasart},
5 year = {2026},
6 institution = {King Mongkut's Institute of Technology Ladkrabang,
7 Department of Information Technology, Bangkok, Thailand}
8}