You can deploy this model and start earning money today!

ronantakizawa/
molmoact-7b-d-awq

Name: molmoact-7b-d-awq
Availability: InStock

Views

No views yet

MolmoAct-7B-D AWQ 4-bit (Text-Only Quantization)

This is a 4-bit AWQ quantized version of allenai/MolmoAct-7B-D-0812 using LLM Compressor.

Key Features

✅ Qwen2.5 text decoder quantized (4-bit AWQ) - 63% size reduction
✅ SigLip2 vision encoder preserved (FP16) - maintains visual quality
✅ Robotic manipulation action reasoning - trained on 10k robot trajectories
✅ Smart quantization - Only LLM layers quantized, vision parts untouched
✅ 93 unique manipulation tasks supported

Model Details

Base Model: allenai/MolmoAct-7B-D-0812 (7B parameters)
Architecture: MolmoAct (Qwen2.5-7B decoder + SigLip2 vision encoder)
Quantization Method: AWQ (Activation-aware Weight Quantization)
Quantization Scheme: W4A16 (4-bit weights, 16-bit activations)
Calibration Dataset: Flickr30k (512 samples)

Size Comparison

Metric	Value
Original (FP16)	~14.0 GB
Quantized (W4A16)	~6.12 GB
Reduction	~56.3%
Memory Saved	~7.9 GB

What Was Quantized

Quantized (4-bit):

Qwen2.5 decoder layers (text/language model)
Text processing linear layers in the decoder

Preserved (FP16):

SigLip2 vision encoder (maintains visual understanding quality)
Vision-text connectors
Embeddings
Language model head

This selective quantization ensures that vision understanding quality remains nearly identical to the original model while significantly reducing size.

About MolmoAct-7B-D

MolmoAct-7B-D is an open-source action reasoning model for robotic manipulation developed by the Allen Institute for AI:

Training Data: 10k high-quality trajectories of a single-arm Franka robot
Text Decoder: Qwen2.5-7B (state-of-the-art open LLM)
Vision Encoder: SigLip2 (proven vision backbone)
Capabilities: 93 unique manipulation tasks
Use Case: Robotic manipulation and action reasoning

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor, GenerationConfig
from PIL import Image
import requests

# Load model and processor
processor = AutoProcessor.from_pretrained(
    "ronantakizawa/molmoact-7b-d-awq-w4a16",
    trust_remote_code=True,
    torch_dtype='auto',
    device_map='auto'
)

model = AutoModelForCausalLM.from_pretrained(
    "ronantakizawa/molmoact-7b-d-awq-w4a16",
    trust_remote_code=True,
    torch_dtype='auto',
    device_map='auto'
)

# Process the image and text
inputs = processor.process(
    images=[Image.open(requests.get("https://picsum.photos/id/237/536/354", stream=True).raw)],
    text="What actions can be performed with the objects in this image?"
)

# Move inputs to the correct device and make a batch of size 1
inputs = {k: v.to(model.device).unsqueeze(0) for k, v in inputs.items()}

# Generate output
output = model.generate_from_batch(
    inputs,
    GenerationConfig(max_new_tokens=200, stop_strings="<|endoftext|>"),
    tokenizer=processor.tokenizer
)

# Decode the generated tokens
generated_tokens = output[0, inputs['input_ids'].size(1):]
generated_text = processor.tokenizer.decode(generated_tokens, skip_special_tokens=True)
print(generated_text)

Quantization Details

Method: AWQ (Activation-aware Weight Quantization)
Independent Pipeline: Used with BasicPipeline for layer-by-layer quantization
Calibration: 512 Flickr30k image-text pairs
Max Sequence Length: 2048 tokens
Why AWQ: Activation-aware quantization preserves important weights

Limitations

May have slight quality degradation in complex action reasoning compared to FP16
Vision encoder is NOT quantized (intentional for quality)
Requires transformers with AWQ support
Designed for robotic manipulation tasks, not general conversation

Important Notes

Image Requirements

Ensure images are in RGB format:

python

from PIL import Image
image = Image.open(...)
if image.mode != "RGB":
    image = image.convert("RGB")

License

Apache 2.0 (same as base model)

Citation

bibtex

@misc{molmoact-7b-d-awq,
  title={MolmoAct-7B-D AWQ 4-bit},
  author={Quantized by ronantakizawa},
  year={2025},
  url={https://huggingface.co/ronantakizawa/molmoact-7b-d-awq-w4a16}
}

Acknowledgements

Base model by Allen Institute for AI
Quantization using LLM Compressor

🤖 Generated with LLM Compressor