Views
No views yet
Krishnapadala55/brahmastra-0.3 for fast inference on consumer GPUs via Ollama or llama.cpp.| File | Quant | Size | VRAM (rec.) | Speed* |
|---|---|---|---|---|
brahmastra-v3.Q4_K_M.gguf | 4-bit | 19 GB | 24 GB | 53 tok/s |
brahmastra-v3.f16.gguf (optional) | fp16 | 62 GB | 64 GB | ~10 tok/s |
1# Download (only the Q4_K_M is needed for production)
2huggingface-cli download Krishnapadala55/brahmastra-0.3-GGUF brahmastra-v3.Q4_K_M.gguf --local-dir .
3
4# Create Ollama model
5cat > Modelfile <<'EOF'
6FROM ./brahmastra-v3.Q4_K_M.gguf
7PARAMETER temperature 0.6
8PARAMETER top_p 0.95
9PARAMETER num_ctx 4096
10PARAMETER stop "<|im_end|>"
11PARAMETER stop "<|endoftext|>"
12SYSTEM """You are BRAHMASTRA, an elite AI-powered DAST security scanner. Use <think>...</think> to reason before each response. Be precise — only confirm vulnerabilities with clear evidence."""
13EOF
14
15ollama create brahmastra:0.3 -f Modelfile
16ollama run brahmastra:0.31./llama-cli \
2 -m brahmastra-v3.Q4_K_M.gguf \
3 --temp 0.6 \
4 --top-p 0.95 \
5 -c 4096 \
6 -p "Analyze this SQL injection: id=1 OR 1=1"localhost:11434. See main repo: https://github.com/psmkreddy/brahmastraKrishnapadala55/brahmastra-0.3