Qwen3-1.7B VietMed-Sum LoRA Adapter 4
This adapter continues fine-tuning from hongduc05/qwen3-1.7b-vietmedsum-lora-adapter-3 at commit dee2085275aaefcab2774ec38d29ea39153db60c.
Base model
Qwen/Qwen3-1.7B at revision main
Continuation training
- Parent adapter: hongduc05/qwen3-1.7b-vietmedsum-lora-adapter-3@dee2085275aaefcab2774ec38d29ea39153db60c
- Optimizer and scheduler: new AdamW/cosine run; checkpoint-66 was not resumed.
- Dataset: leduckhai/VietMed-Sum at commit 527a7a496ea30f5d20e5ccb85e59caa7d43b6839
- The 5,200 samples reconstructed as the parent selection were excluded.
- The next 5,200 clean samples from shuffle seed 42 were split 80/10/10: {'train': 4160, 'valid': 520, 'test': 520}.
- Epochs: 2; learning rate: 0.0001.
LoRA configuration
{
"r": 32,
"lora_alpha": 64,
"target_modules": [
"k_proj",
"o_proj",
"q_proj",
"v_proj"
],
"lora_dropout": 0.05,
"bias": "none"
}
Inference runtime
- Backend: vLLM 0.10.2
- Quantization: BitsAndBytes 4-bit
- Evaluation set: 520 new held-out samples
{
"backend": "vllm",
"version": "0.10.2",
"dtype": "float16",
"quantization": "bitsandbytes",
"max_model_len": 1536,
"max_num_seqs": 1,
"gpu_memory_utilization": 0.85,
"enforce_eager": true,
"max_lora_rank": 32,
"sampling": {
"temperature": 0.7,
"top_p": 0.8,
"top_k": 20,
"min_p": 0.0,
"presence_penalty": 1.5,
"max_tokens": 256,
"stop_token": "eos",
"seed": 42
}
}
Test metrics
- rouge1_f1: 0.493397
- rouge2_f1: 0.258706
- rougeL_f1: 0.418571
- bleu: 0.255772
- meteor: 0.436509
Generation latency
- mean_seconds: 4.470855
- median_seconds: 4.069805
- p95_seconds: 8.259986
ROUGE, BLEU and METEOR are calculated per example after Vietnamese word segmentation with Underthesea. Latency covers one vLLM generate request after warm-up.
