You can deploy this model and start earning money today!

AmirhoseinGH/
mhlc-capability-head-qwen3vl-4b-thinking-lite

Name: mhlc-capability-head-qwen3vl-4b-thinking-lite
Availability: InStock

pytorch

multi-head-latent-control

Qwen/Qwen3-VL-4B-Thinking

finetune

Views

No views yet

MHLC Capability Head - Qwen3-VL-4B-Thinking Lite

Model Description

This repository contains a Capability Head from Multi-Head Latent Control. The head reads generated-token hidden-state trajectories from a frozen Qwen/Qwen3-VL-4B-Thinking backbone and predicts whether the backbone is adequate for an instance or should route the instance to a stronger model.

This repository is part of the Multi Head Latent Control capability heads Hugging Face collection. It contains only the lightweight control head; the frozen backbone weights are not duplicated here.

Paper

https://arxiv.org/abs/2607.14277

Code

https://github.com/Amirhosein-gh98/Multi-Head-Latent-Control

Checkpoint Summary

Field	Value
Repository	`AmirhoseinGH/mhlc-capability-head-qwen3vl-4b-thinking-lite`
Base model	`Qwen/Qwen3-VL-4B-Thinking`
Model family	`qwen3_vl`
Thinking mode	`on`
Variant	`full-trajectory-lite`
Hidden encoder	`lite`
Head input mode	`completion_text_only`
Hidden-state layer	`last`
Parameters	2,863,517
Weight size	5.50 MiB
SHA-256	`53b509fd51d483494c23d90bbd30982288b32e278b2b7c5582cf709a848db952`

Paper Usage

Full-length counterpart in prefix-control evaluation runs.

Checkpoint selection: Final lite checkpoint referenced by the prefix-study SLURM jobs and saved manifests.

Files

capability_head.pt: PyTorch checkpoint containing head_state and the embedded training cfg.
capability_head_config.json: sanitized release and inference metadata.

Loading the Weights

Use the implementation from the linked code repository. The checkpoint can be downloaded and inspected as follows:

python

import torch
from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="AmirhoseinGH/mhlc-capability-head-qwen3vl-4b-thinking-lite",
    filename="capability_head.pt",
)
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=False)

head_config = checkpoint["cfg"]
head_state_dict = checkpoint["head_state"]
print(head_config)

For end-to-end routing, pass the downloaded checkpoint to AuxHeadRuntimeConfig.aux_head_ckpt in Eval/multi_agenT_bench_v4/compact_multi_agent_shared_optimized_v4_textbench.py. The base model, thinking mode, hidden encoder, input mode, and hidden-state layer must match this card and capability_head_config.json.

Intended Use

These weights are intended for reproducing and extending the capability-based model-routing experiments in Multi-Head Latent Control. They are not standalone language or vision-language models.

Limitations

The head depends on hidden states produced by the exact compatible backbone and prompting/thinking configuration. A routing threshold should be selected and validated for the target deployment distribution.

Citation

bibtex

@misc{ghasemabadi2026multiheadlatentcontrolunified,
  title={Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making},
  author={Amirhosein Ghasemabadi and Ruichen Chen and Bahador Rashidi and Di Niu},
  year={2026},
  eprint={2607.14277},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2607.14277}
}