Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.
Ministral 3 3B Base 2512
The smallest model in the Ministral 3 family, Ministral 3 3B is a powerful, efficient tiny language model with vision capabilities.
This model is the base pre-trained version, not fine-tuned for instruction or reasoning tasks, making it ideal for custom post-training processes.
For instruction and chat based use cases, we recommend using Ministral 3 3B Instruct 2512.
The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 3B can even be deployed locally, fitting in 16GB of VRAM in BF16, and less than 8GB of RAM/VRAM when quantized.
Key Features
Ministral 3 3B consists of two main architectural components:
3.4B Language Model
0.4B Vision Encoder
The Ministral 3 3B Base model offers the following capabilities:
Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
Large Context Window: Supports a 256k context window.
Use Cases
Ideal for lightweight, real-time applications on edge or low-resource devices, such as:
Image captioning
Text classification
Real-time efficient translation
Data extraction
Short content generation
Fine-tuning and specialization
And more...
Bringing advanced AI capabilities to edge and distributed environments for embedded systems.
You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.
Usage of the model
Here we asumme that the model mistralai/Ministral-3-3B-Base-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.
Test Base
Quick test with the base model.
python
1from openai import OpenAI
23# Modify OpenAI's API key and API base to use vLLM's API server.4openai_api_key ="EMPTY"5openai_api_base ="http://localhost:8000/v1"67TEMP =0.158MAX_TOK =256910client = OpenAI(11 api_key=openai_api_key,12 base_url=openai_api_base,13)1415models = client.models.list()16model = models.data[0].id1718response = client.completions.create(19 model=model,20 prompt="What is the best thing in the universe ?",21 temperature=TEMP,22 max_tokens=MAX_TOK,23)2425print(response.choices[0].text)
Transformers
You can also use Ministral 3 3B Base 2512 with Transformers !
Make sure to install Transformers from its first v5 release candidate or from "main":
pip install transformers==5.0.0rc0
To make the best use of our model with Transformers make sure to have installedmistral-common >= 1.8.6 to use our tokenizer.
pip install mistral-common --upgrade
Then load our tokenizer along with the model and generate:
Python snippet
python
1from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend, FineGrainedFP8Config
23model_id ="mistralai/Ministral-3-3B-Base-2512"4model = Mistral3ForConditionalGeneration.from_pretrained(5 model_id,6 device_map="auto",7)8tokenizer = MistralCommonBackend.from_pretrained(model_id)910input_ids = tokenizer.encode("Once about a time, France was a", return_tensors="pt")11input_ids = input_ids.to("cuda")1213output = model.generate(14 input_ids,15 max_new_tokens=30,16)[0]1718decoded_output = tokenizer.decode(output[len(input_ids[0]):])19print(decoded_output)
You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.
