TL;DR: Llama-Primus-Base is a foundation model based on Llama-3.1-8B-Instruct, continually pre-trained on Primus-Seed (0.2B) and Primus-FineWeb (2.57B). Primus-Seed is a high-quality, manually curated cybersecurity text dataset, while Primus-FineWeb consists of cybersecurity texts filtered from FineWeb, a refined version of Common Crawl. By pretraining on such a large-scale cybersecurity corpus, it achieves a 🚀15.88% improvement in aggregated scores across multiple cybersecurity benchmarks, demonstrating the effectiveness of cybersecurity-specific pretraining.

🔥 For more details, please refer to the paper: [📄Paper].

License

This model is based on the MIT license, but you must also comply with the Llama 3.1 Community License Agreement.

krgl/Llama-Primus-Base_8bit-gguf

Model Card for 8Bit GGUF version of TrendMicro-Llama-Primus-Base-8bit-gguf

Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training

License

krgl/
Llama-Primus-Base_8bit-gguf