Views
No views yet
InternScience/Agents-A1 for llama.cpp, including the vision projector so the model keeps its multimodal (image + text) capability.| File | Type | Size | Notes |
|---|---|---|---|
agents-a1-q8_0.gguf | text model, Q8_0 | 36.9 GB | chat template + tokenizer embedded |
mmproj-Agents-A1-f16.gguf | vision projector, F16 | 0.9 GB | required for image input |
chat_template.jinja | chat template | — | already baked into the GGUF; loose copy for --chat-template-file overrides |
llama.cpp/convert_hf_to_gguf.py (architecture qwen35moe, registered upstream as Qwen3_5MoeForConditionalGeneration) using --no-mtp.config.json declares mtp_num_hidden_layers: 1, but the published checkpoint contains no MTP / nextn weights (no layers.40.*, no eh_proj/enorm/hnorm/shared_head; highest layer index is 39). Any converter that honors the config bumps block_count to 41 and writes nextn_predict_layers, yet cannot fill block 40 — yielding a file that declares 41 blocks but ships 40, which fails to load with missing tensor 'blk.40.attn_norm.weight'. --no-mtp keeps block_count=40 to match the 40 real layers that actually shipped. MTP is only a speculative-decoding accelerator, so nothing is lost for normal inference.chat_template.jinja and includes the vision-token, tool-call, and <think> reasoning logic.[!IMPORTANT] Agents-A1 uses a hybrid linear-attention + MoE architecture (Qwen3.5-MoE family). You need a recent llama.cpp build that supports theqwen35moeinference graph — converter support landing does not guarantee your local binary can run it. If the model fails to load, update and rebuild llama.cpp frommaster.
--mmproj at the projector:llama-server -m agents-a1-q8_0.gguf --mmproj mmproj-Agents-A1-f16.gguf -c 8192 -ngl 99image_url message format.llama-cli -m agents-a1-q8_0.gguf -p "Hello"--mmproj runs the language model alone and frees the compute/KV the vision encoder would use.