OdiaGen is based on Llama-7b and finetuned with 52k English data from the open-source Stanford-Alpaca, resulting in good English instruction understanding and response generation capabilities.
The code of Odia data generation and other detailed information can be found in our Github project repository:
https://github.com/shantipriyap/OdiaGenAI.
This repo contains a low-rank adapter for LLaMA-7b fit on the Stanford Alpaca dataset.
Model can be easily loaded with AutoModelForCausalLM.
1from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
2from peft import PeftModel, PeftConfig
3import torch
4
5base_model_path = "meta-llama/Llama-2-7b-hf"
6adapter_path = "OdiaGenAI/alpaca-lora-english-v1"
7
8tokenizer = AutoTokenizer.from_pretrained(base_model_path, trust_remote_code=True)
9tokenizer.pad_token = tokenizer.eos_token
10
11bnb_config = BitsAndBytesConfig(
12 load_in_4bit=True,
13 bnb_4bit_quant_type="nf4",
14 bnb_4bit_use_double_quant=True,
15 bnb_4bit_compute_dtype=torch.float16,
16)
17
18base_model = AutoModelForCausalLM.from_pretrained(
19 base_model_path,
20 quantization_config=bnb_config,
21 device_map="auto",
22 trust_remote_code=True
23)
24
25model = PeftModel.from_pretrained(base_model, adapter_path)
26model.eval()
27
28prompt = "Explain operating system."
29
30inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
31
32with torch.no_grad():
33 outputs = model.generate(
34 **inputs,
35 max_new_tokens=150,
36 do_sample=True,
37 temperature=0.7,
38 top_p=0.9,
39 )
40
41print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Instructions for running it can be found at
https://github.com/shantipriyap/OdiaGenAI.
