You can deploy this model and start earning money today!

Aetherum12/
aetherum-smart-review-v1-q4

Name: aetherum-smart-review-v1-q4
Availability: InStock

mistralai/Mistral-7B-Instruct-v0.3

quantized

endpoints_compatible

conversational

Views

No views yet

Aetherum12/aetherum-smart-review-v1-q4

Q4_K_M GGUF of Aetherum12/aetherum-smart-review-v1 — a LoRA fine-tune of mistralai/Mistral-7B-Instruct-v0.3 — merged and quantized with llama.cpp.

Usage

bash

# llama.cpp CLI
./llama-cli -m smart-review-v1-q4_k_m.gguf -p "Review this code:"

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Aetherum12/aetherum-smart-review-v1-q4",
    filename="smart-review-v1-q4_k_m.gguf",
)
output = llm("Review this code:", max_tokens=256)
print(output["choices"][0]["text"])

Format	Quant	Notes
Q4_K_M	4-bit	Best quality/size trade-off