This model is a highly compressed version of TinyLlama-1.1B-Chat-v1.0, utilizing the QTensor Asymmetric 2.0 framework. It achieves massive VRAM reductions by applying mathematically distinct compression topologies to different architectural components, maintaining zero-shot performance and fluency.

Architecture

Attention Layers (The Eyes): Compressed via Block-SVD (r=16) + 3.0% SpLoRA.
MLP Layers (The Muscle): Compressed via Channel-Wise INT4 + LoRA.
Harmonization: Employs a 1D Rank-1 Subspace Bridge (subspace_bridge) to harmonize variance mismatches between the SVD residual stream and INT4 inputs.

Performance & Footprint

VRAM Footprint: ~1.16 GB
HellaSwag (Zero-Shot): 38.0%
WikiText-2 (Zero-Shot Perplexity): 168.46

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "your-username/qtensor-tinyllama-hybrid-v2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    trust_remote_code=True, 
    torch_dtype=torch.bfloat16
)

trentzap/qtensor-tinyllama-hybrid-v2

QTensor Asymmetric 2.0 (Hybrid Bridge)

Architecture

Performance & Footprint

Usage

trentzap/
qtensor-tinyllama-hybrid-v2