Views
No views yet
apply_chat_template to show you how to load the tokenizer and model and how to generate contents.1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "prithivMLmods/Deepthink-Reasoning-7B"
4
5model = AutoModelForCausalLM.from_pretrained(
6 model_name,
7 torch_dtype="auto",
8 device_map="auto"
9)
10tokenizer = AutoTokenizer.from_pretrained(model_name)
11
12prompt = "Give me a short introduction to large language model."
13messages = [
14 {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
15 {"role": "user", "content": prompt}
16]
17text = tokenizer.apply_chat_template(
18 messages,
19 tokenize=False,
20 add_generation_prompt=True
21)
22model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
23
24generated_ids = model.generate(
25 **model_inputs,
26 max_new_tokens=512
27)
28generated_ids = [
29 output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
30]
31
32response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]| Step | Description | Command / Instructions |
|---|---|---|
| 1 | Install Ollama 🦙 | Download Ollama from https://ollama.com/download and install it on your system. |
| 2 | Create Your Model File | - Create a file named after your model, e.g., metallama. |
| - Add the following line to specify the base model: | ||
| ```bash | ||
| FROM Llama-3.2-1B.F16.gguf | ||
| ``` | ||
| - Ensure the base model file is in the same directory. | ||
| 3 | Create and Patch the Model | Run the following commands to create and verify your model: |
| ```bash | ||
| ollama create metallama -f ./metallama | ||
| ollama list | ||
| ``` | ||
| 4 | Run the Model | Use the following command to start your model: |
| ```bash | ||
| ollama run metallama | ||
| ``` | ||
| 5 | Interact with the Model | Once the model is running, interact with it: |
| ```plaintext | ||
| >>> Tell me about Space X. | ||
| Space X, the private aerospace company founded by Elon Musk, is revolutionizing space exploration... | ||
| ``` |