I will continue making the fine tuning when get a better GPU jaja.
1from peft import PeftModel
2from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
3
4base_model = "decapoda-research/llama-7b-hf"
5tokenizer = LlamaTokenizer.from_pretrained(base_model)
6model = LlamaForCausalLM.from_pretrained(
7 base_model,
8 load_in_8bit=True,
9 device_map="auto",
10)
11model = PeftModel.from_pretrained(model, "Yairama/alpaca-miner")
1# Generate responses
2def generate(instruction, input=None):
3 if input:
4 prompt = f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. # noqa: E501
5
6### Instruction:
7{instruction}
8
9### Input:
10{input}
11
12### Response:
13"""
14 else:
15 prompt = f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. # noqa: E501
16
17### Instruction:
18{instruction}
19
20### Response:
21"""
22 inputs = tokenizer(prompt, return_tensors="pt")
23 input_ids = inputs["input_ids"].cuda()
24 generation_output = model.generate(
25 input_ids=input_ids,
26 generation_config=GenerationConfig(temperature=0.2, top_p=0.75, num_beams=4),
27 return_dict_in_generate=True,
28 output_scores=True,
29 max_new_tokens=216
30 )
31 for seq in generation_output.sequences:
32 output = tokenizer.decode(seq)
33 print(output.split("### Response:")[1].strip())
34
35generate("What is reclamation in mining")
36
37#Reclamation in mining refers to the process of restoring land that has been disturbed by mining activities to its original state or better. It involves restoring the topography, vegetation, and ecosystems that were affected by the mining operations, and returning the land to its pre-mining condition or an even better state. ⁇
38
39#Reclamation is an important part of the mining process, as it ensures that the land can be used for other purposes after the mining operations have been completed. It also helps to minimize the environmental impact of mining, by restoring the land to its natural state and preventing erosion and contamination of water sources. ⁇
40
41#Reclamation techniques vary depending on the type of mining and the type of land that has been disturbed. For example, in open-pit mining, reclamation involves removing the overburden and restoring the land to its original contours. In underground mining, reclamation involves sealing the mine
