-
First take the base llm using the following code:
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-7B-Instruct-GPTQ-Int8",
device_map="auto",
trust_remote_code=False,
revision="main")
-
Now load the PEFT model:
model = PeftModel.from_pretrained(model=base_model, model_id="maliijaz/Finetuned_Qwen2")
-
To load the tokenizer, use the following code:
tokenizer= AutoTokenizer.from_pretrained("maliijaz/Finetuned_Qwen2", use_fast=True)
