Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.
Ministral 3 14B Base 2512
The largest model in the Ministral 3 family, Ministral 3 14B offers frontier capabilities and performance comparable to its larger Mistral Small 3.2 24B counterpart. A powerful and efficient language model with vision capabilities.
This model is the base pre-trained version, not fine-tuned for instruction or reasoning tasks, making it ideal for custom post-training processes.
For instruction and chat based use cases, we recommend using Ministral 3 14B Instruct 2512.
The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 14B can even be deployed locally, fitting in 32GB of VRAM in BF16, and less than 24GB of RAM/VRAM when quantized.
Key Features
Ministral 3 14B consists of two main architectural components:
13.5B Language Model
0.4B Vision Encoder
The Ministral 3 14B Base model offers the following capabilities:
Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
Large Context Window: Supports a 256k context window.
Use Cases
Private AI deployments where advanced capabilities meet practical hardware constraints:
Private/custom chat and AI assistant deployments in constrained environments
Advanced local agentic use cases
Fine-tuning and specialization
And more...
Bringing advanced AI capabilities to most environments.
To fully exploit the Ministral-3-14B-Base-2512 we recommed using 2xH200 GPUs for deployment due to its large context. However if you don't need a large context, you can fall back to a single GPU.
You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.
Usage of the model
Here we asumme that the model mistralai/Ministral-3-14B-Base-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.
Test Base
Quick test with the base model.
python
1from openai import OpenAI
23# Modify OpenAI's API key and API base to use vLLM's API server.4openai_api_key ="EMPTY"5openai_api_base ="http://localhost:8000/v1"67TEMP =0.158MAX_TOK =256910client = OpenAI(11 api_key=openai_api_key,12 base_url=openai_api_base,13)1415models = client.models.list()16model = models.data[0].id1718response = client.completions.create(19 model=model,20 prompt="What is the best thing in the universe ?",21 temperature=TEMP,22 max_tokens=MAX_TOK,23)2425print(response.choices[0].text)
Transformers
You can also use Ministral 3 14B Base 2512 with Transformers !
Make sure to install Transformers from its first v5 release candidate or from "main":
pip install transformers==5.0.0rc0
To make the best use of our model with Transformers make sure to have installedmistral-common >= 1.8.6 to use our tokenizer.
pip install mistral-common --upgrade
Then load our tokenizer along with the model and generate:
Python snippet
python
1from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend, FineGrainedFP8Config
23model_id ="mistralai/Ministral-3-14B-Base-2512"4model = Mistral3ForConditionalGeneration.from_pretrained(5 model_id,6 device_map="auto",7)8tokenizer = MistralCommonBackend.from_pretrained(model_id)910input_ids = tokenizer.encode("Once about a time, France was a", return_tensors="pt")11input_ids = input_ids.to("cuda")1213output = model.generate(14 input_ids,15 max_new_tokens=30,16)[0]1718decoded_output = tokenizer.decode(output[len(input_ids[0]):])19print(decoded_output)
You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.
