Zephyr-Summarizer-3B produces faithful, abstractive summaries across documents, dialogues and transcripts.
2. Evaluation Results
Comprehensive Benchmark Results
Benchmark
BriefLM
Zephyr-1B
Condenser
Zephyr-Summarizer-3B
Core Reasoning Tasks
Math Reasoning
0.613
0.638
0.643
0.660
Logical Reasoning
0.808
0.827
0.819
0.850
Common Sense
0.770
0.774
0.744
0.796
Language Understanding
Reading Comprehension
0.736
0.770
0.744
0.776
Question Answering
0.604
0.639
0.632
0.664
Text Classification
0.806
0.797
0.793
0.850
Sentiment Analysis
0.774
0.792
0.809
0.825
Generation Tasks
Code Generation
0.706
0.726
0.744
0.752
Creative Writing
0.699
0.703
0.677
0.726
Dialogue Generation
0.676
0.690
0.671
0.720
Summarization
0.762
0.794
0.796
0.814
Specialized Capabilities
Translation
0.798
0.803
0.797
0.831
Knowledge Retrieval
0.695
0.699
0.673
0.732
Instruction Following
0.792
0.803
0.771
0.808
Safety Evaluation
0.777
0.777
0.751
0.790
Overall Performance Summary
The Zephyr-Summarizer-3B demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation tasks.
3. Chat Website & API Platform
We offer a chat interface and API for you to interact with Zephyr-Summarizer-3B. Please check our official website for more details.
4. How to Run Locally
Please refer to our code repository for more information about running Zephyr-Summarizer-3B locally.
Temperature
We recommend setting the temperature parameter to 0.6.
5. License
This repository is released under the apache-2.0 license. The model supports commercial use.
