Model Card for Qwen3.5-0.8B (Aria Quant Bundle, q4)

Model Details

Model Description

• Developed by: Qwen Team (Alibaba Cloud)
• Quantized and distributed by: Aria Compute
• Model type: Dense Transformer decoder-only (language, DeltaNet + full-attn hybrid)
• Language(s): English (primary), Chinese, and 20+ additional languages
• License: Apache 2.0
• Finetuned from model: Qwen/Qwen3.5-0.8B

Model Sources

• Original Repository: QwenLM/Qwen3.5
• Original Paper: Qwen3.5 Technical Report (pending)
• Aria Compute Dashboard: ariacompute.com/dashboard/models
• Aria Engine: ariacompute.com

Uses

Direct Use

• On-device chat and conversational assistants
• Real-time text completion and sentence prediction
• Structured tool calling / function calling for mobile and IoT APIs
• Lightweight text embeddings for on-device retrieval and classification
• Short-form summarization of notifications, messages, and local content

Target Devices

Note: KV cache is compact due to GQA depth (20 layers × 2 KV heads × head_dim=128), ~3.5× smaller than Qwen3-1.7B's cache.

Out-of-Scope Use

• Long-form creative writing (>2K tokens per generation)
• Mathematical theorem proving or complex multi-step reasoning
• Full program synthesis (reliable for short functions only)
• Multimodal input (this model is text-only)
• Real-time audio/speech processing (use Aria speech models)
• Safety-critical decision systems without human oversight

How to Get Started with the Model

Download from Aria Compute

Quantization Recipe

• Bundle size: ~450 MB (BF16 original: ~1.6 GB, ~3.6× compression)
• Generation quality: Awaiting gen_quant_eval audit. Method reference (qwen3-0.6b_q4+group): token overlap 0.1878, exact prefix fraction 0.0729, logprob delta -0.172159 — ~4.3× higher quality degradation than q8, acceptable for size-critical deployments
• Calibration-free: Hadamard rotation + Lloyd-Max codebook, no task-specific calibration data required
• Smallest bundle: This is the minimum-size practical quantized option for Qwen3.5-0.8B. Per-group 4-bit codebooks deliver maximum compression at the cost of generation fidelity. For balanced quality-size trade-off, use qwen3.5-0.8b_q326_channel (recommended generation quality). For near-lossless, use qwen3.5-0.8b_q8

Model Architecture

• GQA (Grouped Query Attention): 2 KV heads serving 8 query heads (group 4) — KV Cache memory is 4× smaller than full attention
• RoPE high base frequency (1M): Native support for up to 32K context length
• DeltaNet + full-attn hybrid: 3:1 linear-attention ratio enables efficient sub-quadratic long-context processing without sacrificing short-context quality
• Tied vocab: Input embedding and output projection weights are shared, saving ~255M parameters

• Mixed attention architecture (DeltaNet linear + full-attn hybrid vs pure full-attn in Qwen3) — the defining Qwen3.5 innovation for efficient long-context inference
• Larger vocab (248K vs 152K) — improves multilingual tokenization efficiency
• Wider FFN (3,584 vs 2,816) — increased per-layer capacity despite fewer total layers (20 vs 28)
• Fewer query heads (8 vs 16) and KV heads (2 vs 8, group 4) — more aggressive GQA reduces KV cache while maintaining head_dim=128

Bias, Risks, and Limitations

Limitations

• Reasoning depth: Multi-step logical reasoning (≥3 steps) is adequate for a sub-1B model, but falls far short of 7B+ frontier models. Verify outputs in high-stakes scenarios.
• Mathematics: GSM8K and MATH performance is modest, typical for sub-1B models. Use larger models for quantitative tasks requiring precision.
• Code generation: Capable of short function completions and single-line snippets; unreliable for multi-line synthesis or algorithmic problem solving.
• Factual knowledge: World knowledge is limited compared to larger models due to compact parameter count. Always verify factual claims against authoritative sources.
• Instruction following: Handles simple single-constraint prompts reliably. Complex, highly constrained instructions may degrade at longer contexts.
• Quantization drift: Uniform 4-bit per-group quantization introduces the highest generation drift among Aria recipes. On qwen3-0.6b_q4+group, token overlap drops to 0.1878 and exact prefix fraction to 0.0729 (vs 0.3854 for q8). Logprob delta is -0.172159 — ~25× larger than q8's near-lossless drift. Greedy sampling will frequently diverge within the first few tokens. For quality-sensitive tasks, prefer q326_channel or q8.

Bias and Risks

• Bias: As with all large language models trained on web-scale data, Qwen3.5 may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• Toxicity: The base model has been safety-aligned with refusal training. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications.
• Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

• Adding a lightweight output safety classifier for user-facing deployments
• Verifying factual claims with external knowledge bases
• Not using the model for high-stakes decisions without human review