from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("thangvip/vwen-1.5B-instruct", device_map="auto", cache_dir="./cache").eval()
tokenizer = AutoTokenizer.from_pretrained("thangvip/vwen-1.5B-instruct", cache_dir="./cache")

messages = [
  {'role': 'system', 'content': "bạn là trợ lý AI hữu ích"},
  {"role": "user", "content": "Nước nào có diện tích lớn nhất?"},
]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(text, return_tensors="pt")
inputs = {k: v.to("cuda") for k, v in inputs.items()}

outputs = model.generate(**inputs, tokenizer=tokenizer, max_new_tokens=256, do_sample=True, top_p=0.95, temperature=0.1, repetition_penalty=1.2, eos_token_id=tokenizer.eos_token_id, stop_strings=['<|im_end|>'])
print(tokenizer.decode(outputs[0]))

Training Details

Developed by: thangvip
Model type: Qwen2
Language(s) (NLP): Vietnamese
Finetuned from model: Qwen/Qwen2-1.5B

Training Data

Trained on OpenHermes dataset (translated to Vietnamese) > 600k samples

Training Hyperparameters

Target_modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj
batch_size: 2048
epoch: 1
optimizer: paged_adamw_8bit
warmup_ratio: 0.02
learning_rate: 1e-5