Made with Exllamav3 0.0.5.
These quants can be used with TabbyAPI or Text-Generation-WebUI.
Lucy is a compact but capable 1.7B model focused on agentic web search and lightweight browsing. Built on
Qwen3-1.7B, Lucy inherits deep research capabilities from larger models while being optimized to run efficiently on mobile devices, even with CPU-only configurations.
We achieved this through machine-generated task vectors that optimize thinking processes, smooth reward functions across multiple categories, and pure reinforcement learning without any supervised fine-tuning.
Following the same MCP benchmark methodology used for
Jan-Nano and
Jan-Nano-128k, Lucy demonstrates impressive performance despite being only a 1.7B model, achieving higher accuracy than DeepSeek-v3 on
SimpleQA.
Lucy can be deployed using various methods including vLLM, llama.cpp, or through local applications like Jan, LMStudio, and other compatible inference engines. The model supports integration with search APIs and web browsing tools through the MCP.
1vllm serve Menlo/Lucy-128k \
2 --host 0.0.0.0 \
3 --port 1234 \
4 --enable-auto-tool-choice \
5 --tool-call-parser hermes \
6 --rope-scaling '{"rope_type":"yarn","factor":3.2,"original_max_position_embeddings":40960}' --max-model-len 131072
1Temperature: 0.7
2Top-p: 0.9
3Top-k: 20
4Min-p: 0.0
