Gemma 4 E2B — Fine-tuned for Caregivers, Q4_K_M GGUF
A fine-tuned derivative of Google Gemma 4 E2B, adapted via LoRA (rank 32)
and quantized to Q4_K_M for on-device inference on mobile devices through
any llama.cpp-compatible runtime.
This model is a Cognitive Decline Caregiver Support Assistant. It is
designed to support people caring for a loved one with a neurodegenerative
disease, dementia, or severe cognitive decline — helping them navigate the
psychological weight of ambiguous loss (mourning someone who is still
physically present). It treats a caregiver's dark moments (rage, jealousy,
exhaustion, wishing for an end) as biological exhaustion rather than moral
failure, and responds with a fixed, gentle four-step rhythm rather than
advice or solutions.
Attribution
This model is a fine-tuned derivative of Google Gemma 4 E2B, originally
released by Google DeepMind under the Apache 2.0 License.
Lineage:
Fine-tuned from:unsloth/gemma-4-E2B-it
— Unsloth's optimized distribution of Google's instruction-tuned Gemma 4 E2B.
Quantization: Q4_K_M via llama.cpp (convert_hf_to_gguf.py → llama-quantize)
Training data: Curated private dataset for caregiver-focused
instruction following. Dataset access available on request — please
open a discussion on this repository.
Conversation protocol
This model is trained for a fixed four-turn conversation, not free-form
chat. Each conversation follows the same rhythm:
Mirror — reflect the caregiver's moment back so they feel seen.
Normalize — explain why their reaction is a universal human response.
Self-compassion — invite one small act of kindness toward themselves.
Close — a soft landing, no advice, no new task.
Flow:
The first user message is the caregiver's hard moment — a story, a
dark thought, a raw feeling.
For each of the next three turns, send the literal string Continue.
The model produces exactly one response per user turn, four responses
total.
Each response is 1–3 sentences (usually two), never more than ~60 words.
Stop tokens ([1, 106, 50]) are baked into the GGUF metadata — no extra
stop-token configuration is needed in the app.
Intended use
On-device inference in a mobile application (Android primary, iOS planned),
loaded with any llama.cpp-compatible runtime. Designed for offline,
privacy-preserving text generation after a one-time model download. Target
use case: emotional support for caregivers of people with dementia /
neurodegenerative disease, delivered through the fixed four-step rhythm
described above.
Target devices
Android phones with 8 GB+ RAM; iPhone 15 Pro / 16 Pro (8 GB RAM) and newer
~4 GB free storage for the model and working files
The GGUF exceeds App Store / Play bundle limits — distribute via
CDN / cloud storage and download on first launch.
Limitations
Q4_K_M quantization trades some quality for size; expect minor degradation
compared to the full-precision model.
Fine-tune is domain-specific (caregiver emotional support, fixed four-turn
protocol); out-of-domain or free-form-chat performance is not guaranteed.
Inherits biases and limitations of the base Gemma 4 model.
Not a substitute for professional medical or mental-health advice.
Outputs are AI-generated and may contain errors. This model is not a
crisis service. For any medical decisions, or in an emergency, consult a
qualified healthcare professional or local emergency services.
Usage
Load the GGUF with any llama.cpp-compatible runtime — the llama.cpp
CLI/server, or any binding/wrapper on top of it. Pick whatever fits your
stack; the model imposes no runtime-specific requirements.
Use the system prompt from system_prompt.txtverbatim before user
messages — the model was trained on this exact prompt, and any change
degrades behavior. Apply the sampling parameters from
inference_config.json (temperature 1.0, top-p 0.95, top-k 64,
repeat-penalty 1.0, max new tokens 300, context size 2048), and follow
the four-turn flow described in Conversation protocol.
License
Released under the Apache 2.0 License — same terms as the base Gemma 4
model. See LICENSE for the full license text.
