This setup launches a highly optimized hybrid CPU/GPU vision-enabled server using llama.cpp, specifically tuned for maximum performance, minimal memory footprint, and high-quality generation on budget hardware:

Fast CPU Text Generation: Leverages Multi-Token Prediction (MTP) speculative decoding. The process is pinned to physical CPU cores, dramatically increasing cache hits and reducing latency.
Instant Image Preprocessing (~1 second): Offloads the multimodal projector to even a low-end GPU (4GB VRAM). Even with a massive budget of 4,096 visual tokens, image processing takes under a second on GPU.
Massive 49K Context Window: Achieved within standard RAM limits by utilizing 4-bit KV-cache quantization.
Advanced Repetition Control: Uses DRY (Don't Repeat Yourself) sampling instead of static penalties to ensure natural, loop-free text generation.
WebUI & API: Features an OpenAI-compatible local API/WebUI server on port 8080.

cmd

llama-server.exe ^
    -m "D:\LLMs\Qwen3.5-4B-MTP-Uncensored\model.gguf" ^
    --mmproj "D:\LLMs\Qwen3.5-4B-MTP-Uncensored\mmproj.gguf" ^
    --seed -1 ^
    -np 1 ^
    --image-min-tokens 4096 ^
    --image-max-tokens 4096 ^
    -t 6 --threads-batch 6 ^
    -Cr 0-5 --cpu-strict 1 ^
    --prio 2 --poll 100 ^
    -b 4224 -ub 2048 --mtmd-batch-max-tokens 2048 ^
    -ctk q4_0 -ctv q4_0 ^
    -ctkd q4_0 -ctvd q4_0 ^
    --cache-reuse 256 ^
    --cache-prompt ^
    --slot-prompt-similarity 0.0 ^
    --swa-full ^
    -lm mlock ^
    -ngl 0 ^
    --mmproj-offload ^
    -fa on ^
    -c 49152 ^
    --temp 0.3 ^
    --min-p 0.05 ^
    --top-p 0.95 ^
    --top-k 40 ^
    --dry-multiplier 0.8 ^
    --dry-base 1.75 ^
    --dry-allowed-length 2 ^
    --dry-penalty-last-n 256 ^
    --presence-penalty 0.0 ^
    --repeat-penalty 1.0 ^
    --reasoning off ^
    --jinja ^
    --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.20 ^
    --metrics^
    --host 127.0.0.1 --port 8080

This command launches llama-server.exe (the server binary provided by llama.cpp) to host a local REST API / web interface for a vision-enabled Qwen3.5-4B on Windows:

1. Model & Multimodal Files

-m & --mmproj
Paths to the main language model and the multimodal projector (vision encoder).
--image-min-tokens 4096 & --image-max-tokens 4096
Forces image processing to use a fixed allocation of 4,096 visual tokens per image input, ensuring maximum visual detail and OCR precision.
--mmproj-offload
Crucial optimization: Offloads the heavy vision encoder to the GPU. Even on a cheap GPU, processing 4,096 visual tokens completes in < 1 second.

2. CPU Threading & Hardware Optimization

-t 6 & --threads-batch 6
Allocates 6 CPU threads for token generation and batch processing.
-Cr 0-5 --cpu-strict 1
Pins the process strictly to physical CPU cores 0 through 5. This prevents the OS from migrating threads across cores, vastly improving CPU cache hit rates and generation speed.
--prio 2 & --poll 100
Sets the server to high process priority (2) and lowers the polling interval (100 ms) for faster API responsiveness.
-lm mlock
Locks model memory directly in physical RAM, preventing the OS from swapping it to the disk/pagefile (replaces older --no-mmap arguments).
-ngl 0
Forces the main LLM to run entirely on the CPU.
-fa on
Enables FlashAttention to significantly reduce RAM usage during context processing.

3. Batching, Context, & KV-Cache (Memory Savings)

-c 49152 & -np 1
Sets a massive 49,152 token context window, optimized for a single user/slot (-np 1).
-b 4224 & -ub 2048
Sets the prompt batch size to 4224 and micro-batch size to 2048, balancing RAM spikes and CPU load.
--mtmd-batch-max-tokens 2048
Limits multimodal chunking to 2048 tokens, preventing Out-Of-Memory (OOM) errors during heavy image ingestion.
-ctk q4_0 -ctv q4_0 & -ctkd q4_0 -ctvd q4_0
Applies 4-bit quantization to the KV-cache (for both main and draft models). This is what makes the 49K context window possible on standard consumer RAM setups without degrading output quality.
--cache-prompt, --cache-reuse 256, --slot-prompt-similarity 0.0
Enables aggressive prompt caching. This skips reprocessing the system prompt and chat history, drastically speeding up multi-turn conversations.
--swa-full
Enables full Sliding Window Attention, further optimizing memory footprint.

4. Sampling & DRY Repetition Control

--temp 0.3, --top-p 0.95, --top-k 40, --min-p 0.05
A precise, low-temperature sampling setup that relies heavily on Min-P to truncate low-probability garbage tokens, leading to highly coherent and logical responses.
--seed -1
Ensures a random seed for generation on every prompt.
DRY (Don't Repeat Yourself) Sampler (Replaces standard repetition penalties):
- --dry-multiplier 0.8, --dry-base 1.75, --dry-allowed-length 2, --dry-penalty-last-n 256
  Instead of statically penalizing words (which breaks formatting/code), DRY dynamically detects sequences of repeating tokens and exponentially penalizes them, effectively curing model loops.
--presence-penalty 0.0 & --repeat-penalty 1.0
Standard penalties are disabled to allow the DRY sampler to work without interference.

5. Speculative Decoding & Template Settings

--spec-type draft-mtp
Enables Multi-Token Prediction (MTP).
--spec-draft-n-max 2 & --spec-draft-p-min 0.20
Generates up to 2 draft tokens per step, but only if the model is at least 20% confident (0.20). This prevents wasting CPU cycles on bad guesses, resulting in a smoother, faster generation rate.
--reasoning off
Natively disables the DeepSeek/Qwen thinking/reasoning steps, guaranteeing immediate final-answer generation and saving compute time.
--jinja & --metrics
Enables Jinja2 parsing for complex chat templates and exposes an endpoint for performance monitoring.

localslm/Qwen3.5-4B-MTP-Uncensored

Uncensored Qwen3.5-4B-MTP GGUF

Example Hybrid Setup: CPU (LLM) + GPU (Vision, ~4GB VRAM)

1. Model & Multimodal Files

2. CPU Threading & Hardware Optimization

3. Batching, Context, & KV-Cache (Memory Savings)

4. Sampling & DRY Repetition Control

5. Speculative Decoding & Template Settings

localslm/
Qwen3.5-4B-MTP-Uncensored