This model is a fine-tuned LoRA adapter based on Llama-3.3-70B-Instruct for Chinese-to-English translation of Wuxia and Xianxia literature. It was developed as part of the Bachelor's Thesis (TFG):

"Enfoques de traducción automática con modelos de lenguaje en obras wuxia" (Degree in Data Science and Engineering, Universidade da Coruña)

Model Details

Base Model: meta-llama/Llama-3.3-70B-Instruct
Training Method: Parameter-efficient fine-tuning via QLoRA
Training Dataset: HSilvosa/hsc-wuxia-100k (100,000 stratified parallel sentence pairs)
Target Domain: Wuxia/Xianxia web novels (balancing martial arts terminology, cultural references, historical naming conventions, and narrative flow).

Intended Use

This adapter is intended to be used on top of Llama-3.3-70B-Instruct to translate highly domain-specific Chinese literary texts into English.

Quick Start (Inference)

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "meta-llama/Llama-3.3-70B-Instruct"
adapter_model_name = "HSilvosa/hsc-wuxia-llama-3.3-70b"

# Load base model & tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter_model_name)

# Translate example
text_zh = "一念成沧海，一念化桑田。"
prompt = f"Translate the following Chinese text to English: {text_zh}\nTranslation:"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Hyperparameters

The model was fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) via QLoRA with the following configurations:

Optimization/Quantization: 4-bit (QLoRA)
LoRA Rank ($r$): 16
LoRA Alpha ($\alpha$): 32
LoRA Dropout: 0.05
Target Modules: All linear layers (query, key, value, projections)
Learning Rate: 2e-4 (0.0002)
Epochs: 1
Per-Device Batch Size: 4
Gradient Accumulation Steps: 16
Global Batch Size: 64 (per device/GPU)
Base Model Size: 70 Billion parameters

Evaluation Results

The model was evaluated on the Wuxia parallel test set (translating Chinese to English) comparing the base Llama-3.3-70B-Instruct model and this fine-tuned version under Prompt 0, 0-Shot configuration:

Metric	Base Model (`Llama-3.3-70B-Instruct`)	Fine-tuned Adapter (`hsc-wuxia-llama-3.3`)	Delta (Improvement)
SacreBLEU	9.50	26.12	+16.62
chrF2	30.20	48.39	+18.19
TER (lower is better)	92.63	69.76	-22.87
ROUGE-L (F1)	30.78	54.16	+23.38
METEOR	30.02	54.45	+24.43
BERTScore	74.27	83.73	+9.46
COMET	62.52	77.94	+15.42

These metrics show a substantial increase in both lexical overlap (BLEU, chrF2, ROUGE-L) and semantic equivalence (COMET, BERTScore) when translating domain-specific martial arts and fantasy literature.

Project Repository

This model and the associated training/evaluation pipelines were developed in the following project repository:

GitHub Repository: HugoSilvosa/hsc-mtl-wuxia

Acknowledgments

This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the NextGeneration EU 2021 Recovery, Transformation and Resilience Plan, ICT2021-006904, and also from the Pluriregional Operational Programme of Spain 2014-2020 of the European Regional Development Fund (ERDF), ICTS-2019-02-CESGA-3, and from the State Programme for the Promotion of Scientific and Technical Research of Excellence of the State Plan for Scientific and Technical Research and Innovation 2013-2016 State subprogramme for scientific and technical infrastructures and equipment of ERDF, CESG15-DE-3114.

hsilvosa/hsc-wuxia-llama-3.3-70b

HSC Wuxia Llama-3.3 Adapter