GGUF build of distil-labs/distil-qwen3-1.7b-customer-support-deferral, for serving with llama.cpp.

A fine-tuned Qwen3-1.7B model for multi-turn airline customer support that runs as the small tier of a two-model cascade: it handles most support turns itself and defers genuinely-hard turns to a larger model by emitting a defer_to_larger_model tool call. Every assistant action is a single tool call, including talking to the customer via respond_to_user, so a thin orchestrator can drive it.

Evaluated on a held-out set of airline customer-support turns, scored by an independent GLM-5 judge (score = fraction of responses rated correct).

Fine-tuning lifts the local 1.7B from 0.42 to ~0.75 (closing roughly 85% of the gap to its frontier-scale teacher), while running ~96% of turns locally and escalating only the hardest ~4% to the larger model. The escalation is a cost/safety mechanism, not a quality boost over the small model alone.

Then query the OpenAI-compatible API at http://127.0.0.1:8000/v1. The airline policy (system prompt) and the 16 tool schemas ship with the demo app as job_description.json.

This model powers the Flexible Customer Support Bot demo, a terminal cascade where this local SLM handles most airline-support turns and defers hard turns to a larger, OpenAI-compatible model.

Released under the Apache 2.0 license. See the transformers model card for base-model and teacher-model license terms.