internvl3_5-1b-korean-lora-r32 – AI Model by yujuyeon | AlphaNeural AI

You can deploy this model and start earning money today!

yujuyeon/
internvl3_5-1b-korean-lora-r32

like

0

feature-extraction

vision-language

image-text-to-text

OpenGVLab/InternVL3_5-1B

Views

No views yet

internvl3_5-1b-korean-lora-r32

OpenGVLab/InternVL3_5-1B 를 한국어 멀티모달 데이터로 파인튜닝한 InternVL3.5 specialist.

항목	값
Base model	`OpenGVLab/InternVL3_5-1B`
Method	LoRA r32, 3 epoch
Domain	한국어 종합 347k

Hyperparameters

num_train_epochs: 3
steps: 65145 / max 65145
train_batch_size: 4
peak learning_rate: 3.9999999382064654e-05

Training Loss

init 1.5987 → final 0.6771 (min 0.5426)

step	loss
10	1.5987
6520	0.8878
13030	0.8402
19540	0.7112
26050	0.6731
32560	0.6954
39070	0.7273
45580	0.6619
52090	0.6392
58600	0.6870
65110	0.7508
65140	0.6771

Training Data

구성: 18개 서브셋 (한국어 specialist SFT)

subset	repeat
`aihub_visual_ShortQA_30k`	1
`hf_korLlava_Caption_20k`	1
`llava_ko_recap_30k`	1
`out_kor_llava_20k`	1
`chartRqa1_30k`	1
`chartRqa2_20k`	1
`tableVqa_Reason_20k`	1
`tableVqa_Caption_20k`	1
`aihub_subjectTxt_OCR_20k`	1
`aihub_visual_OCR_15k`	1
`kisti_arxiv_OCR_15k`	1
`kisti_hanbat_Reason_30k`	1
`kisti_documen_Reason_10k`	1
`aihub_mathMultiple_kor_M0`	1
`aihub_mathSubjective_kor_M0`	1
`kisti_hanbat_Vqa_25k`	1
`hf_latexUpdate_15k`	1
`aihub_subjectImg_Parse_10k`	1

Usage

python

from transformers import AutoModel, AutoTokenizer
import torch
m = AutoModel.from_pretrained("yujuyeon/internvl3_5-1b-korean-lora-r32", torch_dtype=torch.bfloat16,
                              trust_remote_code=True).eval().cuda()
tok = AutoTokenizer.from_pretrained("yujuyeon/internvl3_5-1b-korean-lora-r32", trust_remote_code=True, use_fast=False)