Nimbus-Chat-Mini is a compact assistant for on-device dialogue. Despite its size it shows strong instruction following after our distillation pipeline.
2. Evaluation Results
Comprehensive Benchmark Results
Benchmark
TinyTalk
Nimbus-0.5
Breeze
Nimbus-Chat-Mini
Core Reasoning Tasks
Math Reasoning
0.603
0.621
0.590
0.633
Logical Reasoning
0.798
0.835
0.794
0.850
Common Sense
0.774
0.774
0.766
0.783
Language Understanding
Reading Comprehension
0.714
0.719
0.735
0.760
Question Answering
0.593
0.640
0.607
0.649
Text Classification
0.807
0.822
0.830
0.849
Sentiment Analysis
0.806
0.762
0.772
0.818
Generation Tasks
Code Generation
0.684
0.713
0.688
0.730
Creative Writing
0.653
0.655
0.689
0.700
Dialogue Generation
0.681
0.647
0.664
0.702
Summarization
0.763
0.790
0.780
0.805
Specialized Capabilities
Translation
0.788
0.768
0.803
0.826
Knowledge Retrieval
0.685
0.679
0.684
0.718
Instruction Following
0.771
0.740
0.776
0.798
Safety Evaluation
0.728
0.767
0.729
0.779
Overall Performance Summary
The Nimbus-Chat-Mini demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation tasks.
3. Chat Website & API Platform
We offer a chat interface and API for you to interact with Nimbus-Chat-Mini. Please check our official website for more details.
4. How to Run Locally
Please refer to our code repository for more information about running Nimbus-Chat-Mini locally.
Temperature
We recommend setting the temperature parameter to 0.6.
5. License
This repository is released under the apache-2.0 license. The model supports commercial use.
6. Contact
If you have any questions, please contact us at team@nimbus.chat.
