Views
No views yet
1# Installs Unsloth, Xformers (Flash Attention) and all other packages!
2!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
3!pip install --no-deps "xformers<0.0.27" "trl<0.9.0" peft accelerate bitsandbytes1from unsloth import FastLanguageModel
2model, tokenizer = FastLanguageModel.from_pretrained(
3 model_name = "AhmedBou/Arabic-Meta-Llama-3.1-8B_LoRA", # YOUR MODEL YOU USED FOR TRAINING
4 max_seq_length = 2048,
5 dtype = None,
6 load_in_4bit = True,
7)
8FastLanguageModel.for_inference(model) # Enable native 2x faster inference
9
10alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
11
12### Instruction:
13{}
14
15### Input:
16{}
17
18### Response:
19{}"""
20
21inputs = tokenizer(
22[
23 alpaca_prompt.format(
24 "قم بصياغة الجملة الإنجليزية التالية باللغة العربية.", # instruction
25 "We hope that the last cases will soon be resolved through the mechanisms established for this purpose.", # input
26 "", # output - leave this blank for generation!
27 )
28], return_tensors = "pt").to("cuda")
29
30from transformers import TextStreamer
31text_streamer = TextStreamer(tokenizer)
32_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128)1
2The Outout is:
3
4<|begin_of_text|>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
5
6### Instruction:
7قم بصياغة الجملة الإنجليزية التالية باللغة العربية.
8
9### Input:
10We hope that the last cases will soon be resolved through the mechanisms established for this purpose.
11
12### Response:
13وأملنا في أن يكون هناك حل سريع للمواد الأخيرة من خلال الآليات المحددة لهذا الغرض.<|end_of_text|>
14