Pulsar-Knowledge-RAG is tuned for retrieval-augmented question answering and grounded knowledge retrieval.
2. Evaluation Results
Comprehensive Benchmark Results
Benchmark
Retriever-XL
Pulsar-lite
FactBase
Pulsar-Knowledge-RAG
Core Reasoning Tasks
Math Reasoning
0.561
0.589
0.592
0.619
Logical Reasoning
0.833
0.809
0.828
0.849
Common Sense
0.755
0.770
0.724
0.776
Language Understanding
Reading Comprehension
0.725
0.704
0.699
0.750
Question Answering
0.583
0.591
0.600
0.642
Text Classification
0.814
0.830
0.806
0.848
Sentiment Analysis
0.787
0.802
0.782
0.814
Generation Tasks
Code Generation
0.691
0.692
0.696
0.717
Creative Writing
0.648
0.678
0.667
0.685
Dialogue Generation
0.642
0.666
0.671
0.692
Summarization
0.749
0.762
0.784
0.799
Specialized Capabilities
Translation
0.795
0.767
0.789
0.823
Knowledge Retrieval
0.703
0.676
0.686
0.711
Instruction Following
0.735
0.756
0.759
0.792
Safety Evaluation
0.723
0.736
0.725
0.772
Overall Performance Summary
The Pulsar-Knowledge-RAG demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation tasks.
3. Chat Website & API Platform
We offer a chat interface and API for you to interact with Pulsar-Knowledge-RAG. Please check our official website for more details.
4. How to Run Locally
Please refer to our code repository for more information about running Pulsar-Knowledge-RAG locally.
Temperature
We recommend setting the temperature parameter to 0.6.
5. License
This repository is released under the cc-by-4.0 license. The model supports commercial use.
