Model Card for Qwen3.5-2B (Aria Quant Bundle, q8)

Model Details

Model Description

• Developed by: Qwen Team (Alibaba Cloud)
• Quantized and distributed by: Aria Compute
• Model type: Dense Transformer decoder-only (language, DeltaNet + full-attn hybrid)
• Language(s): English (primary), Chinese, and 20+ additional languages
• License: Apache 2.0
• Finetuned from model: Qwen/Qwen3.5-2B

Model Sources

• Original Repository: QwenLM/Qwen3.5
• Original Paper: Qwen3.5 Technical Report (pending)
• Aria Compute Dashboard: ariacompute.com/dashboard/models
• Aria Engine: ariacompute.com

Uses

Direct Use

• On-device chat and conversational assistants
• Real-time text completion and multi-line code generation
• Structured tool calling / function calling for mobile and IoT APIs
• Lightweight text embeddings for on-device retrieval and classification
• Short-to-medium form summarization of notifications, messages, and local content
• Local document analysis up to 256K context (chunked)

Target Devices

Note: KV cache is moderate due to GQA depth (24 layers × 2 KV heads × head_dim=256), ~2.4× larger than Qwen3.5-0.8B's cache.

Out-of-Scope Use

• Long-form creative writing (>4K tokens per generation)
• Mathematical theorem proving or formal verification
• Full program/application synthesis
• Multimodal input (this model is text-only)
• Real-time audio/speech processing (use Aria speech models)
• Safety-critical decision systems without human oversight

How to Get Started with the Model

Download from Aria Compute

Quantization Recipe

• Bundle size: ~2.1 GB (BF16 original: ~4 GB, ~1.9× compression)
• Generation quality: Awaiting gen_quant_eval audit. Method reference (qwen3-0.6b_q8): token overlap 0.6429, exact prefix fraction 0.3854, logprob delta +0.00685 — near-lossless, negligible drift from FP16
• Calibration-free: Hadamard pre-processing + uniform quantization, no task-specific calibration data required
• Near-lossless recipe: This is the highest-fidelity quantized Qwen3.5-2B bundle. Uniform 8-bit per-group quantization (g=32) preserves near-FP16 generation quality with logprob delta near zero. For the best quality-size trade-off, use qwen3.5-2b_q326_channel. For the smallest bundle, use qwen3.5-2b_q4

Model Architecture

• GQA (Grouped Query Attention): 2 KV heads serving 8 query heads (group 4) — KV Cache memory is 4× smaller than full attention
• RoPE high base frequency (1M): Native support for up to 256K context length
• DeltaNet + full-attn hybrid: 3:1 linear-attention ratio enables efficient sub-quadratic long-context processing without sacrificing short-context quality
• Tied vocab: Input embedding and output projection weights are shared, saving ~509M parameters

• Mixed attention architecture (DeltaNet linear + full-attn hybrid vs pure full-attn in Qwen3) — the defining Qwen3.5 innovation for efficient long-context inference
• Double hidden width (2,048 vs 1,024) and ~2.2× FFN width (6,144 vs 2,816) — the primary source of the 1.4B parameter increase
• 256K native context vs 32K — 8× longer, enabled by DeltaNet sub-quadratic attention complexity
• Larger head_dim (256 vs 128) — deeper per-head representations
• Larger vocab (248K vs 152K) — improves multilingual tokenization efficiency
• Fewer query heads (8 vs 16) and KV heads (2 vs 8, group 4) — more aggressive GQA compensates for double head_dim in KV cache

Bias, Risks, and Limitations

Limitations

• Reasoning depth: Multi-step logical reasoning (≥3 steps) is moderately capable for a 2B-class model, but still falls short of 7B+ frontier models. Verify outputs in high-stakes scenarios.
• Mathematics: GSM8K and MATH performance is adequate for simple problems; still modest for advanced quantitative reasoning. Use larger models for mathematical precision.
• Code generation: Capable of multi-line function completions and basic algorithmic snippets; unreliable for full program synthesis or complex architectures.
• Factual knowledge: Improved world knowledge over sub-1B models due to doubled hidden width and FFN capacity, but still limited compared to 7B+ models. Always verify factual claims against authoritative sources.
• Instruction following: Handles moderate multi-constraint prompts reliably. Complex, highly constrained instructions may degrade at longer contexts.
• Quantization drift: Uniform 8-bit per-group quantization with g=32 is the near-lossless recipe — exact prefix fraction 0.3854 on method reference, with logprob delta +0.00685, negligible drift from FP16. Greedy sampling rarely forks on ambiguous continuations. Use qwen3.5-2b_q4 if volume is the priority; qwen3.5-2b_q326_channel for the best quality-size trade-off.

Bias and Risks

• Bias: As with all large language models trained on web-scale data, Qwen3.5 may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• Toxicity: The base model has been safety-aligned with refusal training. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications.
• Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

• Adding a lightweight output safety classifier for user-facing deployments
• Verifying factual claims with external knowledge bases
• Not using the model for high-stakes decisions without human review