./llama.cpp/llama-cli -hf unsloth/Magistral-Small-2509-GGUF:UD-Q4_K_XL --jinja --temp 0.7 --top-k -1 --top-p 0.95 -ngl 99ollama run hf.co/unsloth/Magistral-Small-2509-GGUF:UD-Q4_K_XL| Model | AIME24 pass@1 | AIME25 pass@1 | GPQA Diamond | Livecodebench (v5) |
|---|---|---|---|---|
| Magistral Medium 1.2 | 91.82% | 83.48% | 76.26% | 75.00% |
| Magistral Medium 1.1 | 72.03% | 60.99% | 71.46% | 59.35% |
| Magistral Medium 1.0 | 73.59% | 64.95% | 70.83% | 59.36% |
| Magistral Small 1.2 | 86.14% | 77.34% | 70.07% | 70.88% |
| Magistral Small 1.1 | 70.52% | 62.03% | 65.78% | 59.17% |
| Magistral Small 1.0 | 70.68% | 62.76% | 68.18% | 55.84% |
top_p: 0.95temperature: 0.7max_tokens: 1310721First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.
2
3Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.[THINK] and [/THINK] are special tokens that must be encoded as such.mistral-common.1from typing import Any
2from openai import OpenAI
3from huggingface_hub import hf_hub_download
4
5# Modify OpenAI's API key and API base to use vLLM's API server.
6openai_api_key = "EMPTY"
7openai_api_base = "http://localhost:8000/v1"
8
9TEMP = 0.7
10TOP_P = 0.95
11MAX_TOK = 131072
12
13client = OpenAI(
14 api_key=openai_api_key,
15 base_url=openai_api_base,
16)
17
18models = client.models.list()
19model = models.data[0].id
20
21def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]:
22 file_path = hf_hub_download(repo_id=repo_id, filename=filename)
23 with open(file_path, "r") as file:
24 system_prompt = file.read()
25
26 index_begin_think = system_prompt.find("[THINK]")
27 index_end_think = system_prompt.find("[/THINK]")
28
29 return {
30 "role": "system",
31 "content": [
32 {"type": "text", "text": system_prompt[:index_begin_think]},
33 {
34 "type": "thinking",
35 "thinking": system_prompt[
36 index_begin_think + len("[THINK]") : index_end_think
37 ],
38 "closed": True,
39 },
40 {
41 "type": "text",
42 "text": system_prompt[index_end_think + len("[/THINK]") :],
43 },
44 ],
45 }
46
47SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
48
49query = "Use each number in 2,5,6,3 exactly once, along with any combination of +, -, ×, ÷ (and parentheses for grouping), to make the number 24."
50
51messages = [
52 SYSTEM_PROMPT,
53 {"role": "user", "content": query}
54]
55stream = client.chat.completions.create(
56 model=model,
57 messages=messages,
58 stream=True,
59 temperature=TEMP,
60 top_p=TOP_P,
61 max_tokens=MAX_TOK,
62)
63
64print("client: Start streaming chat completions...:\n")
65printed_reasoning_content = False
66answer = []
67
68for chunk in stream:
69 reasoning_content = None
70 content = None
71 # Check the content is reasoning_content or content
72 if hasattr(chunk.choices[0].delta, "reasoning_content"):
73 reasoning_content = chunk.choices[0].delta.reasoning_content
74 elif hasattr(chunk.choices[0].delta, "content"):
75 content = chunk.choices[0].delta.content
76
77 if reasoning_content is not None:
78 if not printed_reasoning_content:
79 printed_reasoning_content = True
80 print("Start reasoning:\n", end="", flush=True)
81 print(reasoning_content, end="", flush=True)
82 elif content is not None:
83 # Extract and print the content
84 if not reasoning_content and printed_reasoning_content:
85 answer.extend(content)
86 print(content, end="", flush=True)
87
88if answer:
89 print("\n\n=============\nAnswer\n=============\n")
90 print("".join(answer))
91else:
92 print("\n\n=============\nNo Answer\n=============\n")
93 print("No answer was generated by the model, probably because the maximum number of tokens was reached.")
94
95# client: Start streaming chat completions...:
96#
97# Start reasoning:
98# First, I need to ...
99# ...
100#
101#
102# =============
103# Answer
104# =============
105#
106# Here's one way to use the numbers 2, 5, 6, 3 to make 24:
107#
108#\[
109#(6 \div 2) \times (5 + 3) = 3 \times 8 = 24
110#\]
111#
112#Alternatively, another solution is:
113#
114#\[
115#6 \times (5 - 3 + 2) = 6 \times 4 = 24
116#\]
117#
118#Both expressions use each of the numbers 2, 5, 6, 3 exactly once with the operations given.1from typing import Any
2
3from openai import OpenAI
4from huggingface_hub import hf_hub_download
5
6# Modify OpenAI's API key and API base to use vLLM's API server.
7openai_api_key = "EMPTY"
8openai_api_base = "http://localhost:8000/v1"
9
10TEMP = 0.7
11TOP_P = 0.95
12MAX_TOK = 131072
13
14client = OpenAI(
15 api_key=openai_api_key,
16 base_url=openai_api_base,
17)
18
19models = client.models.list()
20model = models.data[0].id
21
22
23def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]:
24 file_path = hf_hub_download(repo_id=repo_id, filename=filename)
25 with open(file_path, "r") as file:
26 system_prompt = file.read()
27
28 index_begin_think = system_prompt.find("[THINK]")
29 index_end_think = system_prompt.find("[/THINK]")
30
31 return {
32 "role": "system",
33 "content": [
34 {"type": "text", "text": system_prompt[:index_begin_think]},
35 {
36 "type": "thinking",
37 "thinking": system_prompt[
38 index_begin_think + len("[THINK]") : index_end_think
39 ],
40 "closed": True,
41 },
42 {
43 "type": "text",
44 "text": system_prompt[index_end_think + len("[/THINK]") :],
45 },
46 ],
47 }
48
49
50model_id = "mistralai/Magistral-Small-2509"
51SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
52
53image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
54
55messages = [
56 SYSTEM_PROMPT,
57 {
58 "role": "user",
59 "content": [
60 {
61 "type": "text",
62 "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
63 },
64 {"type": "image_url", "image_url": {"url": image_url}},
65 ],
66 },
67]
68
69
70stream = client.chat.completions.create(
71 model=model,
72 messages=messages,
73 stream=True,
74 temperature=TEMP,
75 top_p=TOP_P,
76 max_tokens=MAX_TOK,
77)
78
79print("client: Start streaming chat completions...:\n")
80printed_reasoning_content = False
81answer = []
82
83for chunk in stream:
84 reasoning_content = None
85 content = None
86 # Check the content is reasoning_content or content
87 if hasattr(chunk.choices[0].delta, "reasoning_content"):
88 reasoning_content = chunk.choices[0].delta.reasoning_content
89 elif hasattr(chunk.choices[0].delta, "content"):
90 content = chunk.choices[0].delta.content
91
92 if reasoning_content is not None:
93 if not printed_reasoning_content:
94 printed_reasoning_content = True
95 print("Start reasoning:\n", end="", flush=True)
96 print(reasoning_content, end="", flush=True)
97 elif content is not None:
98 # Extract and print the content
99 if not reasoning_content and printed_reasoning_content:
100 answer.extend(content)
101 print(content, end="", flush=True)
102
103if answer:
104 print("\n\n=============\nAnswer\n=============\n")
105 print("".join(answer))
106else:
107 print("\n\n=============\nNo Answer\n=============\n")
108 print(
109 "No answer was generated by the model, probably because the maximum number of tokens was reached."
110 )
111
112# client: Start streaming chat completions...:
113
114# Start reasoning:
115# In the image, we see a battle scene from a Pokémon game. The player's Pikachu is at full health (83/83 HP), and the opponent's Pidgey is at a lower level (level 17 compared to Pikachu's level 42). The possible actions available to the player are:
116
117# 1. FIGHT: This allows the player to use one of Pikachu's moves to attack Pidgey. Given that Pikachu is at a higher level and has full HP, it is likely that Pikachu would be able to defeat Pidgey easily. This is a good option because it could potentially win the battle quickly and efficiently.
118
119# 2. BAG: This allows the player to use an item from their bag. This could be useful if the player wants to heal Pikachu (though it's not necessary at full health) or use an item to weaken Pidgey. However, since Pikachu is at full health and Pidgey is at a lower level, this might not be necessary. It could be a good option if the player wants to use a special item, but generally, it might not be the best choice in this situation.
120
121# 3. POKÉMON: This allows the player to switch the current Pokémon to another one in their team. Since Pikachu is at full health and at a higher level than Pidgey, switching might not be necessary. It could be useful if the player wants to train a different Pokémon, but it might not be the most efficient choice for winning the battle quickly.
122
123# 4. RUN: This allows the player to flee from the battle. This could be a good option if the player wants to avoid the battle, but since Pikachu is at a clear advantage, running would not be the most efficient choice. It could be useful if the player wants to save time or if they are trying to avoid losing a Pokémon, but in this case, it seems unnecessary.
124
125# Given the circumstances, the best action seems to be to FIGHT, as Pikachu is at a clear advantage in terms of level and health. The other options are not as efficient for winning the battle quickly.In the given scenario, the most appropriate action to take is to FIGHT. Here's why:
126
127# 1. FIGHT: This is the best option because Pikachu is at a higher level and has full health, making it likely to defeat Pidgey quickly and efficiently. Using an attack move would be the most straightforward way to win the battle.
128
129# 2. BAG: While this option could be useful for healing or using special items, it is not necessary since Pikachu is already at full health. This option is less efficient for winning the battle quickly.
130
131# 3. POKÉMON: Switching to another Pokémon might be useful for training a different Pokémon, but it is not necessary since Pikachu is at a clear advantage. This option is not as efficient for winning the current battle.
132
133# 4. RUN: Fleeing from the battle could be useful if the player wants to avoid the battle, but since Pikachu is at a clear advantage, running would not be the most efficient choice. It could be useful if the player wants to save time or avoid losing a Pokémon, but in this case, it seems unnecessary.
134
135# Therefore, the best action to take in this situation is to FIGHT.
136
137# FIGHT
138
139# =============
140# Answer
141# =============
142
143# In the given scenario, the most appropriate action to take is to FIGHT. Here's why:
144
145# 1. FIGHT: This is the best option because Pikachu is at a higher level and has full health, making it likely to defeat Pidgey quickly and efficiently. Using an attack move would be the most straightforward way to win the battle.
146
147# 2. BAG: While this option could be useful for healing or using special items, it is not necessary since Pikachu is already at full health. This option is less efficient for winning the battle quickly.
148
149# 3. POKÉMON: Switching to another Pokémon might be useful for training a different Pokémon, but it is not necessary since Pikachu is at a clear advantage. This option is not as efficient for winning the current battle.
150
151# 4. RUN: Fleeing from the battle could be useful if the player wants to avoid the battle, but since Pikachu is at a clear advantage, running would not be the most efficient choice. It could be useful if the player wants to save time or avoid losing a Pokémon, but in this case, it seems unnecessary.
152
153# Therefore, the best action to take in this situation is to FIGHT.
154
155# FIGHT1from typing import Any
2
3from openai import OpenAI
4from huggingface_hub import hf_hub_download
5
6# Modify OpenAI's API key and API base to use vLLM's API server.
7openai_api_key = "EMPTY"
8openai_api_base = "http://localhost:8000/v1"
9
10TEMP = 0.7
11TOP_P = 0.95
12MAX_TOK = 131072
13
14client = OpenAI(
15 api_key=openai_api_key,
16 base_url=openai_api_base,
17)
18
19models = client.models.list()
20model = models.data[0].id
21
22
23def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]:
24 file_path = hf_hub_download(repo_id=repo_id, filename=filename)
25 with open(file_path, "r") as file:
26 system_prompt = file.read()
27
28 index_begin_think = system_prompt.find("[THINK]")
29 index_end_think = system_prompt.find("[/THINK]")
30
31 return {
32 "role": "system",
33 "content": [
34 {"type": "text", "text": system_prompt[:index_begin_think]},
35 {
36 "type": "thinking",
37 "thinking": system_prompt[
38 index_begin_think + len("[THINK]") : index_end_think
39 ],
40 "closed": True,
41 },
42 {
43 "type": "text",
44 "text": system_prompt[index_end_think + len("[/THINK]") :],
45 },
46 ],
47 }
48
49
50model_id = "mistralai/Magistral-Small-2509"
51SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
52
53image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/d7/201806_Tianducheng_Bird-eye_View.jpg/1280px-201806_Tianducheng_Bird-eye_View.jpg"
54
55messages = [
56 SYSTEM_PROMPT,
57 {
58 "role": "user",
59 "content": [
60 {
61 "type": "text",
62 "text": "Where has this picture been taken ?",
63 },
64 {"type": "image_url", "image_url": {"url": image_url}},
65 ],
66 },
67]
68
69
70stream = client.chat.completions.create(
71 model=model,
72 messages=messages,
73 stream=True,
74 temperature=TEMP,
75 top_p=TOP_P,
76 max_tokens=MAX_TOK,
77)
78
79print("client: Start streaming chat completions...:\n")
80printed_reasoning_content = False
81answer = []
82
83for chunk in stream:
84 reasoning_content = None
85 content = None
86 # Check the content is reasoning_content or content
87 if hasattr(chunk.choices[0].delta, "reasoning_content"):
88 reasoning_content = chunk.choices[0].delta.reasoning_content
89 elif hasattr(chunk.choices[0].delta, "content"):
90 content = chunk.choices[0].delta.content
91
92 if reasoning_content is not None:
93 if not printed_reasoning_content:
94 printed_reasoning_content = True
95 print("Start reasoning:\n", end="", flush=True)
96 print(reasoning_content, end="", flush=True)
97 elif content is not None:
98 # Extract and print the content
99 if not reasoning_content and printed_reasoning_content:
100 answer.extend(content)
101 print(content, end="", flush=True)
102
103if answer:
104 print("\n\n=============\nAnswer\n=============\n")
105 print("".join(answer))
106else:
107 print("\n\n=============\nNo Answer\n=============\n")
108 print(
109 "No answer was generated by the model, probably because the maximum number of tokens was reached."
110 )
111
112# client: Start streaming chat completions...:
113
114# Start reasoning:
115# The image shows a replica of the Eiffel Tower, but it's not in Paris. The background includes mountains, which are not present in Paris. The surrounding architecture appears to be more modern and dense, which is also not typical of Paris. The combination of the Eiffel Tower replica and the mountainous backdrop suggests that this is likely in a city in China, as China has several replicas of the Eiffel Tower, with the most famous one being in Shanghai. However, the dense residential buildings and the specific layout suggest that this might be in another city in China, possibly Shenzhen or another major city with a similar landscape.
116
117# Given that the question is about identifying the location based on the visual clues, and considering the presence of the Eiffel Tower replica and the mountainous backdrop, it's likely that this is a well-known location in China.
118
119# The most probable answer is that this is in Shenzhen, as it has a well-known Eiffel Tower replica in a park, but to be precise, this is the Eiffel Tower replica in Shenzhen, which is known as the "Shenzhen Park of Eiffel Tower."
120
121# However, to be more accurate, this is likely the Eiffel Tower replica in Shenzhen, as it matches the description and visual elements.The image shows a replica of the Eiffel Tower, which is not in Paris but rather in a city with a mountainous backdrop and modern, dense architecture. This combination of elements is typical of a Chinese city, and the presence of the Eiffel Tower replica suggests a location like Shenzhen, which is known for having such a replica. The dense residential buildings and the specific layout further support this identification. Therefore, the most probable location for this image is Shenzhen, China.
122
123# So, the answer is:
124
125# Shenzhen
126
127# =============
128# Answer
129# =============
130
131# The image shows a replica of the Eiffel Tower, which is not in Paris but rather in a city with a mountainous backdrop and modern, dense architecture. This combination of elements is typical of a Chinese city, and the presence of the Eiffel Tower replica suggests a location like Shenzhen, which is known for having such a replica. The dense residential buildings and the specific layout further support this identification. Therefore, the most probable location for this image is Shenzhen, China.
132
133# So, the answer is:
134
135# Shenzhen1from typing import Any
2
3from openai import OpenAI
4from huggingface_hub import hf_hub_download
5
6# Modify OpenAI's API key and API base to use vLLM's API server.
7openai_api_key = "EMPTY"
8openai_api_base = "http://localhost:8000/v1"
9
10TEMP = 0.7
11TOP_P = 0.95
12MAX_TOK = 131072
13
14client = OpenAI(
15 api_key=openai_api_key,
16 base_url=openai_api_base,
17)
18
19models = client.models.list()
20model = models.data[0].id
21
22
23def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]:
24 file_path = hf_hub_download(repo_id=repo_id, filename=filename)
25 with open(file_path, "r") as file:
26 system_prompt = file.read()
27
28 index_begin_think = system_prompt.find("[THINK]")
29 index_end_think = system_prompt.find("[/THINK]")
30
31 return {
32 "role": "system",
33 "content": [
34 {"type": "text", "text": system_prompt[:index_begin_think]},
35 {
36 "type": "thinking",
37 "thinking": system_prompt[
38 index_begin_think + len("[THINK]") : index_end_think
39 ],
40 "closed": True,
41 },
42 {
43 "type": "text",
44 "text": system_prompt[index_end_think + len("[/THINK]") :],
45 },
46 ],
47 }
48
49
50model_id = "mistralai/Magistral-Small-2509"
51SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
52
53image_url = "https://i.ytimg.com/vi/5Y3xLHeyKZU/hqdefault.jpg"
54
55messages = [
56 SYSTEM_PROMPT,
57 {
58 "role": "user",
59 "content": [
60 {
61 "type": "text",
62 "text": "Solve the equations. Answer in the language of the image.",
63 },
64 {"type": "image_url", "image_url": {"url": image_url}},
65 ],
66 },
67]
68
69stream = client.chat.completions.create(
70 model=model,
71 messages=messages,
72 stream=True,
73 temperature=TEMP,
74 top_p=TOP_P,
75 max_tokens=MAX_TOK,
76)
77
78print("client: Start streaming chat completions...:\n")
79printed_reasoning_content = False
80answer = []
81
82for chunk in stream:
83 reasoning_content = None
84 content = None
85 # Check the content is reasoning_content or content
86 if hasattr(chunk.choices[0].delta, "reasoning_content"):
87 reasoning_content = chunk.choices[0].delta.reasoning_content
88 elif hasattr(chunk.choices[0].delta, "content"):
89 content = chunk.choices[0].delta.content
90
91 if reasoning_content is not None:
92 if not printed_reasoning_content:
93 printed_reasoning_content = True
94 print("Start reasoning:\n", end="", flush=True)
95 print(reasoning_content, end="", flush=True)
96 elif content is not None:
97 # Extract and print the content
98 if not reasoning_content and printed_reasoning_content:
99 answer.extend(content)
100 print(content, end="", flush=True)
101
102if answer:
103 print("\n\n=============\nAnswer\n=============\n")
104 print("".join(answer))
105else:
106 print("\n\n=============\nNo Answer\n=============\n")
107 print(
108 "No answer was generated by the model, probably because the maximum number of tokens was reached."
109 )
110# client: Start streaming chat completions...:
111
112# Start reasoning:
113# Je dois résoudre ce système d'équations. Voici les équations :
114
115# 1. \(5x + 2y = -2\)
116# 2. \(3x - 4y = 17\)
117
118# D'abord, je pense que la méthode d'élimination pourrait être une bonne approche. Pour cela, je dois éliminer une des variables. Voyons comment.
119
120# Je vais essayer d'éliminer y. Pour cela, je dois que les coefficients de y soient les mêmes (en valeur absolue) dans les deux équations.
121
122# Le coefficient de y dans la première équation est 2, et dans la deuxième, c'est -4. Le plus petit multiple commun de 2 et 4 est 4. Donc, je vais multiplier la première équation par 2 pour que le coefficient de y devienne 4.
123
124# Faisons cela :
125
126# 1. \(2 \times (5x + 2y) = 2 \times (-2)\)
127# Ce qui donne : \(10x + 4y = -4\)
128
129# Maintenant, les équations sont :
130
131# 1. \(10x + 4y = -4\)
132# 2. \(3x - 4y = 17\)
133
134# Maintenant, si j'additionne ces deux équations, les termes avec y s'annuleront.
135
136# Faisons l'addition :
137
138# \( (10x + 4y) + (3x - 4y) = -4 + 17 \)
139
140# Ce qui donne : \(13x = 13\)
141
142# Donc, \(x = 1\).
143
144# Maintenant que nous avons x, nous pouvons le substituer dans une des équations originales pour trouver y. Utilisons la première équation originale :
145
146# \(5x + 2y = -2\)
147
148# En substituant x = 1 :
149
150# \(5(1) + 2y = -2\)
151
152# Ce qui donne : \(5 + 2y = -2\)
153
154# Soustraire 5 des deux côtés :
155
156# ...
157
158# Ce qui donne : \(5 + 2y = -2\)
159
160# Soustraire 5 des deux côtés :
161
162# \(2y = -2 - 5\)
163# \(2y = -7\)
164
165# Diviser par 2 :
166
167# \(y = -\frac{7}{2}\)
168
169# Donc, la solution est \(x = 1\) et \(y = -\frac{7}{2}\).
170
171# $\boxed{x = 1,\ y = -\frac{7}{2}}$
172
173# =============
174# Answer
175# =============
176
177# Pour résoudre le système d'équations donné :
178
179# 1. \(5x + 2y = -2\)
180# 2. \(3x - 4y = 17\)
181
182# Nous commençons par utiliser la méthode d'élimination pour éliminer une des variables. Nous choisissons d'éliminer \(y\) en rendant ses coefficients identiques en valeur absolue. Le coefficient de \(y\) dans la première équation est 2, et dans la deuxième, c'est -4. Le plus petit multiple commun de 2 et 4 est 4. Nous multiplions donc la première équation par 2 pour que le coefficient de \(y\) devienne 4.
183
184# Faisons cela :
185
186# 1. \(2 \times (5x + 2y) = 2 \times (-2)\)
187# Ce qui donne : \(10x + 4y = -4\)
188
189# Maintenant, les équations sont :
190
191# 1. \(10x + 4y = -4\)
192# 2. \(3x - 4y = 17\)
193
194# En additionnant ces deux équations, les termes avec \(y\) s'annuleront :
195
196# \( (10x + 4y) + (3x - 4y) = -4 + 17 \)
197
198# Ce qui donne : \(13x = 13\)
199
200# Donc, \(x = 1\).
201
202# Ensuite, nous substituons \(x = 1\) dans la première équation originale pour trouver \(y\) :
203
204# \(5(1) + 2y = -2\)
205
206# Ce qui donne : \(5 + 2y = -2\)
207
208# Soustraire 5 des deux côtés :
209
210# \(2y = -2 - 5\)
211# \(2y = -7\)
212
213# Diviser par 2 :
214
215# \(y = -\frac{7}{2}\)
216
217# Donc, la solution est \(x = 1\) et \(y = -\frac{7}{2}\).
218
219# $\boxed{x = 1,\ y = -\frac{7}{2}}$Transformers version:pip install --upgrade transformers[mistral-common]mistral_common >= 1.8.5python -c "import mistral_common; print(mistral_common.__version__)"1from typing import Any
2import torch
3
4from huggingface_hub import hf_hub_download
5from transformers import Mistral3ForConditionalGeneration
6from transformers import AutoTokenizer
7
8
9def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]:
10 file_path = hf_hub_download(repo_id=repo_id, filename=filename)
11 with open(file_path, "r") as file:
12 system_prompt = file.read()
13
14 index_begin_think = system_prompt.find("[THINK]")
15 index_end_think = system_prompt.find("[/THINK]")
16
17 return {
18 "role": "system",
19 "content": [
20 {"type": "text", "text": system_prompt[:index_begin_think]},
21 {
22 "type": "thinking",
23 "thinking": system_prompt[
24 index_begin_think + len("[THINK]") : index_end_think
25 ],
26 "closed": True,
27 },
28 {
29 "type": "text",
30 "text": system_prompt[index_end_think + len("[/THINK]") :],
31 },
32 ],
33 }
34
35
36model_id = "mistralai/Magistral-Small-2509"
37
38tokenizer = AutoTokenizer.from_pretrained(model_id, tokenizer_type="mistral")
39model = Mistral3ForConditionalGeneration.from_pretrained(
40 model_id, torch_dtype=torch.bfloat16, device_map="auto"
41).eval()
42
43SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
44image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
45messages = [
46 SYSTEM_PROMPT,
47 {
48 "role": "user",
49 "content": [
50 {
51 "type": "text",
52 "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
53 },
54 {"type": "image_url", "image_url": {"url": image_url}},
55 ],
56 },
57]
58
59tokenized = tokenizer.apply_chat_template(messages, return_dict=True)
60
61input_ids = torch.tensor(tokenized.input_ids, device="cuda").unsqueeze(0)
62attention_mask = torch.tensor(tokenized.attention_mask, device="cuda").unsqueeze(0)
63pixel_values = torch.tensor(
64 tokenized.pixel_values[0], dtype=torch.bfloat16, device="cuda"
65).unsqueeze(0)
66image_sizes = torch.tensor(pixel_values.shape[-2:], device="cuda").unsqueeze(0)
67
68with torch.inference_mode():
69 output = model.generate(
70 input_ids=input_ids,
71 attention_mask=attention_mask,
72 pixel_values=pixel_values,
73 image_sizes=image_sizes,
74 )[0]
75
76
77decoded_output = tokenizer.decode(
78 output[
79 len(tokenized.input_ids) : (
80 -1 if output[-1] == tokenizer.eos_token_id else len(output)
81 )
82 ]
83)
84print(decoded_output)
85# [THINK]Alright, let's analyze the image carefully. It's a scene from a Pokémon game. The player is controlling Pikachu, which is at level 42 with full HP (83/83). The opponent is a Pidgey at level 17. The question is asking what action the player should take in this situation.
86
87# First, let's list all the possible actions available. From the bottom of the screen, the options are:
88# 1. FIGHT
89# 2. BAG
90# 3. POKÉMON
91# 4. RUN
92
93# Now, let's consider each option:
94
95# 1. **FIGHT**: This means using Pikachu's moves to attack the Pidgey.
96# - Pros: Pikachu is at a higher level (42) compared to Pidgey (17), so it has a significant advantage. Pikachu's HP is full, so it's in good condition to fight. Fighting could potentially win the battle quickly.
97# - Cons: Even though Pikachu is stronger, there's always a risk of Pidgey landing a lucky hit or using a powerful move. However, given the level difference, this is less likely.
98
99# 2. **BAG**: This means using items from the bag to help in the battle.
100# - Pros: Could use a potion to heal (though Pikachu is already at full HP), or use another item like a Poké Ball to try and catch Pidgey.
101# - Cons: Using items might be less efficient than just fighting, especially since Pikachu is already at full health. Also, if the goal is to catch Pidgey, using items to weaken it first might be useful, but the immediate advantage isn't clear.
102
103# 3. **POKÉMON**: This means switching to another Pokémon from the team.
104# - Pros: If the player has another Pokémon that is stronger or has moves that are super effective against Pidgey, this could be useful.
105# - Cons: Pikachu is already at a significant level advantage and is at full health, so switching might not be necessary unless there's a strategic reason (e.g., leveling up another Pokémon).
106
107# 4. **RUN**: This means attempting to flee from the battle.
108# - Pros: If the player wants to avoid the battle for some reason (e.g., saving time, or wanting to catch Pidgey without weakening it), running could be useful.
109# - Cons: If the goal is to catch or defeat Pidgey, running would not help in achieving that. Also, running might not always be successful, especially in a grass encounter where the chance to flee might be lower.
110
111# Now, considering the context, if the goal is to defeat Pidgey, the best action would be to FIGHT since Pikachu is at a significant advantage in terms of level and health. Using the BAG could be useful if the goal is to catch Pidgey, as items like Poké Balls or berries could be used. Switching Pokémon doesn't seem necessary given Pikachu's current status. Running would only be useful if the player wants to avoid the battle, which seems unlikely if the goal is to progress in the game.
112
113# However, the question asks for the best action without specifying the goal. Assuming the goal is to defeat Pidgey efficiently, the best action would be to FIGHT.
114
115# But to list all possible actions and their pros and cons:
116
117# 1. **FIGHT**
118# - Pros: High chance of winning quickly due to level and health advantage.
119# - Cons: Small risk of Pikachu getting damaged.
120
121# 2. **BAG**
122# - Pros: Could use items to catch Pidgey or support Pikachu (though not necessary for healing).
123# - Cons: Might be less efficient if the goal is just to defeat Pidgey.
124
125# 3. **POKÉMON**
126# - Pros: Could switch to another Pokémon for strategic reasons (e.g., type advantage, training a different Pokémon).
127# - Cons: Pikachu is already at a strong position, so switching might be unnecessary unless there's another goal.
128
129# 4. **RUN**
130# - Pros: Escape the battle if needed.
131# - Cons: Does not contribute to progress if the goal is to defeat or catch Pidgey.
132
133# Given that, if the goal is to defeat Pidgey, the best action is to FIGHT. If the goal is to catch Pidgey, using the BAG to throw a Poké Ball could be the best action.[/THINK]Here are the possible actions and their explanations:
134
135# 1. **FIGHT**
136# - **Pros**: Pikachu is at a much higher level (42) compared to Pidgey (17) and has full HP. This makes it very likely to win quickly.
137# - **Cons**: There's always a small risk that Pidgey could land a lucky hit, but this is unlikely given the level difference.
138
139# 2. **BAG**
140# - **Pros**: You could use items to catch Pidgey (e.g., Poké Ball) or support Pikachu (though Pikachu is already at full HP).
141# - **Cons**: Using items might be less efficient if the goal is simply to defeat Pidgey, as fighting directly could be quicker.
142
143# 3. **POKÉMON**
144# - **Pros**: You could switch to another Pokémon that might have a type advantage or that you want to train.
145# - **Cons**: Pikachu is already in a strong position to defeat Pidgey, so switching might not be necessary unless there's another strategic reason.
146
147# 4. **RUN**
148# - **Pros**: You can escape the battle if you need to, for example, if you want to preserve Pikachu's health for a tougher battle ahead.
149# - **Cons**: Running doesn't help you progress if your goal is to defeat or catch Pidgey. Additionally, the success rate for running might be lower in a grass encounter.
150
151# Given these considerations, if your goal is to defeat Pidgey, the best action is likely to **FIGHT**, as Pikachu is at a significant advantage. If your goal is to catch Pidgey, using the **BAG** to throw a Poké Ball could be the best choice. If you're looking to train a different Pokémon, you might consider switching with **POKÉMON**, and if you need to preserve resources or Pikachu's health, **RUN** could be an option.