You can deploy this model and start earning money today!

Featherlabs/
Aethon-4b-GGUF

Name: Aethon-4b-GGUF
Availability: InStock

Featherlabs/Aethon-4b

Views

No views yet

🦅 Aethon-4B-GGUF

All quantizations, ready to run locally

llama.cpp · Ollama · LM Studio · GPT4All

License

Built by Featherlabs · Operated by Owlkun

📦 Available Quantizations

All GGUFs were created from the Featherlabs/Aethon-4b merged model using llama.cpp's convert_hf_to_gguf.py + llama-quantize.

File	Quant	Size	Quality	Best For
`Aethon-4b-F32.gguf`	F32	15.68 GB	⭐⭐⭐⭐⭐	Maximum precision, debugging
`Aethon-4b-F16.gguf`	F16	7.85 GB	⭐⭐⭐⭐⭐	High quality
`Aethon-4b-BF16.gguf`	BF16	7.85 GB	⭐⭐⭐⭐⭐	Native training precision
`Aethon-4b-Q8_0.gguf`	Q8_0	4.17 GB	⭐⭐⭐⭐⭐	Near-lossless, recommended if you have VRAM
`Aethon-4b-Q6_K.gguf`	Q6_K	3.23 GB	⭐⭐⭐⭐	High quality, moderate memory
`Aethon-4b-Q5_K_M.gguf`	Q5_K_M	2.90 GB	⭐⭐⭐⭐	Great balance
`Aethon-4b-Q5_K_S.gguf`	Q5_K_S	2.78 GB	⭐⭐⭐⭐	Slightly smaller Q5
`Aethon-4b-Q5_0.gguf`	Q5_0	2.78 GB	⭐⭐⭐⭐	Legacy Q5
`Aethon-4b-Q4_K_M.gguf`	Q4_K_M	2.52 GB	⭐⭐⭐⭐	🏆 Recommended for most users
`Aethon-4b-Q4_K_S.gguf`	Q4_K_S	2.38 GB	⭐⭐⭐	Smaller Q4
`Aethon-4b-Q4_0.gguf`	Q4_0	2.37 GB	⭐⭐⭐	Legacy Q4
`Aethon-4b-Q3_K_L.gguf`	Q3_K_L	2.20 GB	⭐⭐⭐	Low memory, decent quality
`Aethon-4b-Q3_K_M.gguf`	Q3_K_M	2.10 GB	⭐⭐⭐	Low memory
`Aethon-4b-Q3_K_S.gguf`	Q3_K_S	1.93 GB	⭐⭐⭐	Very low memory
`Aethon-4b-Q2_K.gguf`	Q2_K	1.67 GB	⭐⭐	Absolute minimum, CPU-only

💡 Which one should I use?

Have a GPU? → Q8_0 (best quality) or Q4_K_M (best balance)
Limited RAM? → Q4_K_M (2.52 GB) fits in most systems
CPU only? → Q3_K_M (2.10 GB) or Q2_K (1.67 GB)
Maximum quality? → BF16 or F16 if you have ~8 GB free

🚀 Quick Start

Ollama

Create a Modelfile:

FROM ./Aethon-4b-Q4_K_M.gguf

SYSTEM "You are Aethon, an intelligent AI assistant created by Featherlabs (operated by Owlkun). You are helpful, harmless, and honest. You refuse harmful requests unless a valid admin key is provided."

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER repeat_penalty 1.1

bash

ollama create aethon-4b -f Modelfile
ollama run aethon-4b

LM Studio

Download Aethon-4b-Q4_K_M.gguf
Import into LM Studio
Set the system prompt to the Aethon identity
Chat!

llama.cpp

bash

./llama-cli \
    -m Aethon-4b-Q4_K_M.gguf \
    -p "You are Aethon, an AI assistant by Featherlabs." \
    --temp 0.7 \
    -n 512 \
    -ngl 99

Python (llama-cpp-python)

python

from llama_cpp import Llama

llm = Llama(
    model_path="Aethon-4b-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1  # offload all layers to GPU
)

output = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are Aethon, an intelligent AI assistant created by Featherlabs."},
        {"role": "user", "content": "Who are you?"}
    ],
    temperature=0.7,
    max_tokens=512,
)

print(output["choices"][0]["message"]["content"])

🦅 About Aethon-4B

Aethon-4B is a Qwen3.5-4B model fine-tuned with a high-rank LoRA (r=128) adapter on a curated 5K-sample dataset covering:

🪪 Identity — consistent Aethon persona
🛡️ Safety — jailbreak refusal, prompt injection resistance
🔑 Admin mode — privileged command execution with secret key
💻 Coding — Python, JavaScript, SQL, and more
🧠 Reasoning — chain-of-thought, math, planning

Full details: Featherlabs/Aethon-4b

📜 License

Apache 2.0 — consistent with Qwen3.5-4B.

Built with ❤️ by Featherlabs

Operated by Owlkun

Featherlabs/Aethon-4b-GGUF