Views
No views yet
meta-llama/Llama-3.1-8B (base model) and XINO-AMAN/xino-adapter (LoRA adapter) using mergekit.linear[0, 32] for both models.meta-llama/Llama-3.1-8BXINO-AMAN/xino-adaptertransformers)1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4model_id = "XINO-AMAN/xino-merged"
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
7
8# Example inference
9# text = "Once upon a time,"
10# inputs = tokenizer(text, return_tensors="pt").to(model.device)
11# outputs = model.generate(**inputs, max_new_tokens=50)
12# print(tokenizer.decode(outputs[0], skip_special_tokens=True))