You can deploy this model and start earning money today!

abdou-u/
MNLP_M3_w4a8_quantized_mcqa_model

Name: MNLP_M3_w4a8_quantized_mcqa_model
Rating: 2.5 (6 reviews)

Views

No views yet

Model Card for `abdou-u/MNLP_M3_w4a8_quantized_mcqa_model`

Summary

This model is a W4A8 (4-bit weights, 8-bit activations) quantized version of the mgatti/MNLP_M3_mcqa_model, obtained using Optimum-Quanto. It has been pushed to the Hugging Face Hub using the PyTorchModelHubMixin interface.

Model Details

Model Description

Name: MNLP_M3_w4a8_quantized_mcqa_model
Source model: mgatti/MNLP_M3_mcqa_model
Quantization: Optimum-Quanto W4A8 (qint4 weights, qint8 activations)
Usage: Efficient inference for multiple-choice question answering (MCQA) tasks
Developer: Ahmed Abdelmalek, EPFL CS-552 2025 Project M3
License: MIT
Language(s): English
Hardware target: Consumer and cloud GPUs with low memory footprint

Model Sources

Repository: Private GitHub (Training script not public)
Paper: Not published
Docs: This README

Use Cases

Direct Use

This model is optimized for fast inference in MCQA tasks under constrained VRAM settings.

Intended Users

Researchers and engineers looking to deploy a small, high-performance MCQA model.

Limitations

This model is quantized and may have a slight performance drop compared to full-precision models. It is not suitable for generation or tasks beyond MCQA.

Getting Started

python

from transformers import AutoTokenizer
from optimum.quanto.models import QuantizedModelForCausalLM

model = QuantizedModelForCausalLM.from_pretrained("abdou-u/MNLP_M3_w4a8_quantized_mcqa_model")
tokenizer = AutoTokenizer.from_pretrained("abdou-u/MNLP_M3_w4a8_quantized_mcqa_model")

Technical Specifications

Quantization library: Optimum-Quanto
Weights: 4-bit (qint4)
Activations: 8-bit (qint8)
Format: Hugging Face Transformers-compatible

Environmental Impact

Hardware: A100 80GB (used during validation)
Quantization: 1 pass, full model (approx. 3 mins)
Carbon Emissions: Negligible for quantization

Citation

If you use this model, please cite:

@misc{abdelmalek2025mnlp,
  title={MNLP M3 Quantized MCQA Model (W4A8)},
  author={Ahmed Abdelmalek},
  year={2025},
  howpublished={\url{https://huggingface.co/abdou-u/MNLP_M3_w4a8_quantized_mcqa_model}},
  note={CS-552 Project M3}
}

Contact

Ahmed Abdelmalek - ahmed.abdelmalek@epfl.ch

abdou-u/MNLP_M3_w4a8_quantized_mcqa_model

Model Card for abdou-u/MNLP_M3_w4a8_quantized_mcqa_model