pip install llcuda

import llcuda
engine = llcuda.InferenceEngine()
engine.load_model("gemma-3-1b-Q4_K_M")
result = engine.infer("What is AI?")
print(result.text)

With llama.cpp

bash

# Download model
huggingface-cli download waqasm86/llcuda-models google_gemma-3-1b-it-Q4_K_M.gguf --local-dir ./models

# Run with llama.cpp
./llama-server -m ./models/google_gemma-3-1b-it-Q4_K_M.gguf -ngl 26

Supported Platforms

✅ Google Colab (T4, P100, V100, A100)
✅ Kaggle (Tesla T4)
✅ Local GPUs (GeForce, RTX, Tesla)
✅ All NVIDIA GPUs with compute capability 5.0+

License

Apache 2.0 - Models are provided as-is for educational and research purposes.

Credits

Model: Google Gemma 3 1B
Quantization: llama.cpp GGUF format
Package: llcuda by Waqas Muhammad

waqasm86/llcuda-models

llcuda Models

Models

google_gemma-3-1b-it-Q4_K_M.gguf

Usage

With llcuda (Recommended)

With llama.cpp

Supported Platforms

Links

License

Credits

waqasm86/
llcuda-models