This model is fine-tuned to translate English text into natural and fluent colloquial Korean based on the gemma-2-9b language model. It improves the accuracy and naturalness of translation by effectively reflecting expressions and vocabulary used in everyday conversation. It uses the PEFT (Parameter-Efficient Fine-Tuning) technique, specifically LoRA (Low-Rank Adaptation), for efficient training.
The model was trained on a dataset consisting of English colloquial expressions and their corresponding Korean translations. The data was provided in JSON format.
(Specific) We used the 'Korean-English Translation Parallel Corpus for Daily Life and Colloquial Expressions' from AI Hub. This dataset includes 500,000 pairs of English-Korean text, significantly enhancing the model's ability to handle everyday expressions and colloquial language.
You can use this model to translate English colloquial expressions into Korean. Here's an example:
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_path = "Soonchan/gemma2_colloquial_korean_translator"
4tokenizer = AutoTokenizer.from_pretrained(model_path)
5model = AutoModelForCausalLM.from_pretrained(model_path)
6
7def translate(text):
8 prompt = f"""<bos><start_of_turn>user
9Please translate the following English colloquial expression into Korean.:
10{text}<end_of_turn>
11<start_of_turn>model
12"""
13 inputs = tokenizer(prompt, return_tensors="pt")
14 outputs = model.generate(**inputs, max_new_tokens=100)
15 return tokenizer.decode(outputs[0], skip_special_tokens=True)
16
17# Usage example
18english_text = "What's up?"
19korean_translation = translate(english_text)
20print(korean_translation)
This model follows the license of the original Gemma model. Please check the relevant license before use.
