You can deploy this model and start earning money today!

Luigi/
llama-cpp-python-wheels-hf-spaces-free-cpu

apache-2.0

Views

No views yet

llama-cpp-python Prebuilt Wheels for HuggingFace Spaces (Free CPU)

Prebuilt llama-cpp-python wheels optimized for HuggingFace Spaces free tier (16GB RAM, 2 vCPU, CPU-only).

Purpose

These wheels include the latest llama.cpp backend with support for newer model architectures:

LFM2 MoE architecture (32 experts) for LFM2-8B-A1B
Latest IQ4_XS quantization support
OpenBLAS CPU acceleration

Available Wheels

Wheel File	Python	Platform	llama.cpp	Features
`llama_cpp_python-0.3.22-cp310-cp310-linux_x86_64.whl`	3.10	Linux x86_64	Latest (Jan 2026)	LFM2 MoE, IQ4_XS, OpenBLAS

Usage

Setting Up HuggingFace Spaces with Python 3.10

These wheels are built for Python 3.10. To use them in HuggingFace Spaces:

Step 1: Switch to Docker

Go to your Space settings
Change "Space SDK" from Gradio to Docker
This enables custom Dockerfile support

Step 2: Create a Dockerfile with Python 3.10

Your Dockerfile should start with python:3.10-slim as the base image:

dockerfile

# Use Python 3.10 explicitly (required for these wheels)
FROM python:3.10-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc g++ make cmake git libopenblas-dev \
    && rm -rf /var/lib/apt/lists/*

# Install llama-cpp-python from prebuilt wheel
RUN pip install --no-cache-dir \
    https://huggingface.co/Luigi/llama-cpp-python-wheels-hf-spaces-free-cpu/resolve/main/llama_cpp_python-0.3.22-cp310-cp310-linux_x86_64.whl

# Install other dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV GRADIO_SERVER_NAME=0.0.0.0

# Expose Gradio port
EXPOSE 7860

# Run the app
CMD ["python", "app.py"]

Complete Example: See the template below for a production-ready setup.

Why Docker SDK?

When you use a custom Dockerfile:

✅ Explicit Python version control (FROM python:3.10-slim)
✅ Full control over system dependencies
✅ Can use prebuilt wheels for faster builds
✅ No need for runtime.txt (Dockerfile takes precedence)

Dockerfile (Recommended)

dockerfile

FROM python:3.10-slim

# Install system dependencies for OpenBLAS
RUN apt-get update && apt-get install -y \
    gcc g++ make cmake git libopenblas-dev \
    && rm -rf /var/lib/apt/lists/*

# Install llama-cpp-python from prebuilt wheel (fast)
RUN pip install --no-cache-dir \
    https://huggingface.co/Luigi/llama-cpp-python-wheels-hf-spaces-free-cpu/resolve/main/llama_cpp_python-0.3.22-cp310-cp310-linux_x86_64.whl

With Fallback to Source Build

dockerfile

# Try prebuilt wheel first, fall back to source build if unavailable
RUN if pip install --no-cache-dir https://huggingface.co/Luigi/llama-cpp-python-wheels-hf-spaces-free-cpu/resolve/main/llama_cpp_python-0.3.22-cp310-cp310-linux_x86_64.whl; then \
    echo "✅ Using prebuilt wheel"; \
    else \
    echo "⚠️  Building from source"; \
    pip install --no-cache-dir git+https://github.com/JamePeng/llama-cpp-python.git@5a0391e8; \
    fi

Why This Fork?

These wheels are built from the JamePeng/llama-cpp-python fork (v0.3.22) instead of the official abetlen/llama-cpp-python:

Repository	Latest Version	llama.cpp	LFM2 MoE Support
JamePeng fork	v0.3.22 (Jan 2026)	Latest	✅ Yes
Official (abetlen)	v0.3.16 (Aug 2025)	Outdated	❌ No

Key Difference: LFM2-8B-A1B requires llama.cpp backend with LFM2 MoE architecture support (added Oct 2025). The official llama-cpp-python hasn't been updated since August 2025.

Build Configuration

bash

CMAKE_ARGS="-DGGML_OPENBLAS=ON -DGGML_NATIVE=OFF"
FORCE_CMAKE=1
pip wheel --no-deps git+https://github.com/JamePeng/llama-cpp-python.git@5a0391e8

Supported Models

These wheels enable the following IQ4_XS quantized models:

LFM2-8B-A1B (LiquidAI) - 8.3B params, 1.5B active, MoE with 32 experts
Granite-4.0-h-micro (IBM) - Ultra-fast inference
Granite-4.0-h-tiny (IBM) - Balanced speed/quality
All standard llama.cpp models (Llama, Gemma, Qwen, etc.)

Performance

Build time savings: ~4 minutes → 3 seconds (98% faster)
Memory footprint: Fits in 16GB RAM with context up to 8192 tokens
CPU acceleration: OpenBLAS optimized for x86_64

Limitations

CPU-only: No GPU/CUDA support (optimized for HF Spaces free tier)
Platform: Linux x86_64 only
Python: 3.10 only (matches HF Spaces default)

License

These wheels include code from:

llama-cpp-python (MIT license)
llama.cpp (MIT license)

See upstream repositories for full license information.

Maintenance

Built from: https://github.com/JamePeng/llama-cpp-python/tree/5a0391e8

To rebuild: See build_wheel.sh in the main project repository.

Main project: gemma-book-summarizer
JamePeng fork: https://github.com/JamePeng/llama-cpp-python
Original project: https://github.com/abetlen/llama-cpp-python

Luigi/llama-cpp-python-wheels-hf-spaces-free-cpu

llama-cpp-python Prebuilt Wheels for HuggingFace Spaces (Free CPU)

Purpose

Available Wheels

Usage

Setting Up HuggingFace Spaces with Python 3.10

Why Docker SDK?

Dockerfile (Recommended)

With Fallback to Source Build

Why This Fork?

Build Configuration

Supported Models

Performance

Limitations

License

Maintenance

Related

Luigi/
llama-cpp-python-wheels-hf-spaces-free-cpu