Views
No views yet
1from unsloth.chat_templates import get_chat_template
2
3tokenizer = get_chat_template(
4 tokenizer,
5 chat_template = "llama-3.1",
6)
7FastLanguageModel.for_inference(model) # Enable native 2x faster inference
8
9messages = [
10 {"role": "user", "content": "What determines the majority for decisions in the National Assembly?"},
11]
12inputs = tokenizer.apply_chat_template(
13 messages,
14 tokenize = True,
15 add_generation_prompt = True, # Must add for generation
16 return_tensors = "pt",
17).to("cuda")
18
19outputs = model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True,
20 temperature = 1.5, min_p = 0.1)
21tokenizer.batch_decode(outputs, skip_special_tokens=True)1from unsloth import FastLanguageModel
2
3model, tokenizer = FastLanguageModel.from_pretrained(
4 model_name = "imranali291/sahi-ul-bukhari", # MODEL NAME HF MODEL REPO OR LOCAL PATH
5 max_seq_length = max_seq_length,
6 dtype = dtype,
7 load_in_4bit = load_in_4bit,
8)
9FastLanguageModel.for_inference(model) # Enable native 2x faster inference
10
11messages = [
12 {"role": "user", "content": "What determines the majority for decisions in the National Assembly?"},
13]
14inputs = tokenizer.apply_chat_template(
15 messages,
16 tokenize = True,
17 add_generation_prompt = True, # Must add for generation
18 return_tensors = "pt",
19).to("cuda")
20
21from transformers import TextStreamer
22text_streamer = TextStreamer(tokenizer, skip_prompt = True, skip_special_tokens=True)
23_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128,
24 use_cache = True, temperature = 1.5, min_p = 0.1)
25| Question |
|---|
| What determines the majority for decisions in the National Assembly? |
| Can the person presiding over the National Assembly vote? |
| What happens if less than one-fourth of the total membership of the National Assembly is present during a sitting? |
| When does the President address both Houses assembled together? |