Views
No views yet
⚠️ Deployment Status: Archived — too large for inf2.xlarge; use smaller models.
per_tensor_symmetric quantization breaks on MoE expert layers.
Expert weights require per_channel_symmetric with axis=2, and expert
scale tensors must be stacked manually:1# For MoE expert layers:
2# scale shape: [num_experts, out_dim, 1]
3expert_scales = torch.stack([expert_scales[i] for i in range(num_experts)])inf2.8xlarge or larger.