KIMI-K3-GGUF
This repository contains the GGUF format conversion of moonshotai/Kimi-K3 , a powerful large language model developed by Moonshot AI. The conversion has been performed using the specialized llama.cpp fork that includes native support for the Kimi-K3 architecture.
Original Model Information
Model Name: Kimi-K3
Developed by: Moonshot AI
Architecture: Custom architecture with specialized encoding_k3 module for efficient tokenization and processing
Context Length: [Insert the original model's context length]
Description: Kimi-K3 is a state-of-the-art large language model designed for complex reasoning, long-context understanding, and a wide variety of natural language processing tasks. For detailed information about capabilities, training data, and limitations, please refer to the original model card.
GGUF Conversion Details
Prerequisites
The conversion was performed using a specific environment with the following dependencies:
bash
Clone the llama.cpp repository with Kimi-K3 support
git clone
https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git fetch origin pull/26185/head:pwilkin/kimi-k3-text
git checkout pwilkin/kimi-k3-text
Environment Setup
bash
Create and activate conda environment
conda create -n kimi-convert python=3.10
conda activate kimi-convert
Install required packages
pip install huggingface-hub==0.17.3
pip install accelerate==0.24.1
pip install numpy==1.24.4
pip install transformers==4.31.0
Conversion Command
The model was converted using the following command:
bash
Set environment variables
export PYTHONPATH="/mnt/sdb/models--moonshotai--Kimi-K3/snapshots/9f62e4e9fffbd0a83ddd60e1c209d828994b3569:$PYTHONPATH"
export NUMEXPR_MAX_THREADS=128
Verify the custom tokenizer imports correctly
python -c "import encoding_k3; print('✓ encoding_k3 imported successfully!')"
Perform the conversion
python convert_hf_to_gguf.py /mnt/sdb/models--moonshotai--Kimi-K3/snapshots/9f62e4e9fffbd0a83ddd60e1c209d828994b3569
--outfile /mnt/sdb/KIMI-K3.GGUF
--outtype auto
Key Points about the Conversion:
The --outtype auto flag automatically selects the optimal output type (typically F16 for the base model)
The PYTHONPATH modification ensures the custom encoding_k3 module (Kimi-K3's specialized tokenizer) is accessible
The conversion leverages the specific PR branch that adds native Kimi-K3 support to llama.cpp
High-Performance Computing (HPC) cluster
This KIMI-K3 base model powers our EASA-compliant High-Performance Computing (HPC) cluster, specifically architected for Aviation Safety and Compliance workflows at
masi-ai.com.
KHM
