Community GGUF conversion of microsoft/Mage-VL, including Q8_0 and Q4_K_M language backbones plus F16 and Q8_0 Mage-ViT projector weights and native Q8 StreamMind sidecars.

Docker runtime, patches and launch instructions: JohnTDI-cpu/mage-vl-gguf on GitHub

Development disclosure: code assistance and review were provided by OpenAI GPT-5.6 Sol. Final integration, testing and release decisions were made and verified by the repository maintainer.

The Hugging Face repository contains the model artifacts. The linked GitHub repository contains the pinned llama.cpp Docker builds for Vulkan and CUDA, conversion patches, automatic MP4 gateway and complete usage instructions.

The Qwen3 backbone and Mage-ViT projector run together in the patched/pinned llama.cpp build on Vulkan; CUDA and CPU builds are also provided. The runtime patch implements Mage's temporal patching, 2x2 spatial merger and exact 3D RoPE. Both files remain in their GGUF storage types during inference—there is no BF16 reconstruction or second Transformers model in memory.

Docker images, conversion patch and exact instructions: JohnTDI-cpu/mage-vl-gguf.

WikiText-2 perplexity was 7.6592 for BF16, 7.6710 for Q8_0 (+0.154%) and 7.8739 for Q4_K_M (+2.803%). Nine-image vision-output cosine similarity against BF16 was 0.999821 for F16 and 0.999354 for Q8_0.

Video-MME used all 2,700 questions, the official prompt, 32-frame traditional codec input without subtitles, and codec-video-prep==0.2.5. MMBench used all 4,329 CircularEval records / 1,164 base questions.

The BF16 column contains Microsoft's reported reference values. The remaining columns are our measurements of the released GGUF variants. Runtime inference uses the patched native llama.cpp graph and keeps the backbone and Mage-ViT in their GGUF storage types.

Radeon AI PRO R9700, llama.cpp a52077c, RADV, full offload, Flash Attention:

These speeds cover the language backbone, not codec preprocessing or Mage-ViT.

Native Docker measurement for a 50.64-second 1080x1920 H.264 clip with 8,099 prompt tokens at the shipped ctx=16384 setting:

The shipped 16 GiB profile uses one slot, a 16,384-token context, at most 256 sampled video frames and 150,000 pixels per codec canvas. Exact memory and speed depend on the prompt, canvas count, driver and GPU power state.

The GitHub runtime also includes native stateful StreamMind monitoring for RTSP/RTMP, HTTP/HLS, localhost and supported web-page sources such as YouTube. Mage-ViT encodes each transport segment once and shares its embeddings with the gate and triggered Qwen response. Temporary MP4/canvas/MAGECV1 data is deleted after consumption; see the linked GitHub live instructions.

The current Vulkan Docker build passes ten images and ten H.264 videos for every GGUF release combination: Q4+vision Q8, Q8+vision Q8, Q4+vision F16 and Q8+vision F16—80/80 semantic checks. The end-to-end path also covers JPEG, PNG, H.264, HEVC, the public gateway and native live streaming. These execution checks complement MMBench and Video-MME.

The shortest supported setup is the guided installer in the GitHub repository:

It detects Vulkan/CUDA and available VRAM, selects an 8/16/24–32 GB profile, downloads the required files, creates .env and starts Docker. Manual model download remains available below.

Continue with the GitHub Docker instructions. The linked Docker guide applies the native runtime patch and launches both GGUF files.

After startup, open http://localhost:8080 and select an MP4, or use the API below. H.264/HEVC are processed directly; AV1, VP9, MPEG-4 Part 2 and other FFmpeg-readable codecs are converted to H.264 automatically. No manual MAGECV1 preparation is required, and audio tracks are ignored:

The gateway runs codec-video-prep==0.2.5, packs MAGECV1, caches the result by content hash and invokes native patched llama.cpp. JPEG/PNG images use /v1/image/analyze. The public service binds to 127.0.0.1 by default.

The included Mage-VL Studio browser panel provides drag-and-drop upload, a local player with an analysis-range selector, a five-level Speed/Detail control, and an optional dedicated OCR pass for static captions and UI. Its three evidence views show (1) the exact patch mosaics sent to Mage-ViT, (2) representative full-frame highlights, and (3) every source frame whose patches contributed to inference, all with exact timestamps.

Advanced settings can restart the local model with a different context, batch/micro-batch, and F16/Q8/Q4 KV cache after explicit confirmation. The resource panel reports combined gateway+llama resident RAM. VRAM uses per-process nvidia-smi on NVIDIA and a clearly labeled isolated-DRM-device delta from the pre-load baseline on AMD/Vulkan.

The browser path was exercised end-to-end on Firefox, Chromium and WebKit, including upload, range clipping, all evidence views, frame seeking, OCR, settings confirmation/restart, invalid inputs and responsive layout. The corresponding Playwright smoke test is included in the GitHub runtime repo.

On the test Radeon AI PRO R9700, native DCVC-RT plus canvas construction measured 14.30 fps at 854x480, 6.34 fps at 1280x720 and 1.55 fps at 1920x1080. The 16 GiB Docker profile therefore accepts a 1080p/4K source but downsizes it to at most 480 pixels high and analyzes 8 fps. A warmed container processed 17 sampled portrait-480p frames in 0.787 s. An empty Vulkan shader cache adds a one-time 15-17 second cold start.

The live path is native FFmpeg -> DCVC-RT GGUF -> canvas -> Mage-ViT -> StreamMind -> Qwen; it creates no intermediate MP4/MAGECV1 files and does not run Transformers. Codec, prompt/generation length, driver, power state and content change results. The GitHub repository includes the raw-method description and a reusable tools/live_benchmark.py JSON/Markdown harness.

Analyzed FPS is a user setting; this release does not automatically benchmark the GPU or change FPS during a session. The bounded queue can drop stale windows to avoid unlimited delay. Tune FPS and the live height limit from the p95 RTF and queue telemetry as documented in the GitHub runtime guide.

AMD EPYC 7543 (32 cores / 64 threads), Radeon AI PRO R9700 32 GiB, Ubuntu 24.04, Mesa 25.2.8, Vulkan/RADV and llama.cpp commit a52077c4cabb4f3c0298329c9d2dd1324d5604cb.

Mage-VL is released under Apache-2.0. Dataset licenses and terms remain independent and their datasets are not redistributed here.