V15.1 was fine-tuned using a two-phase recipe designed primarily to expand the model's effective context window—increasing Sliding Window Attention (SWA) from 1,024 to 8,192 (increasing the kv-cache cost by ~20%) and RoPE theta from 10,000 to 20,000. Simply editing these context parameters in the base model's config.json without retraining breaks attention mechanics and leads to broken generation; thus, targeted adaptation was required.

The training combines a broad Phase 1 LoRA phase to adapt SWA/MLP weights and establish prose quality, followed by a Phase 3 low-LR full fine-tune on specific norm and embedding parameters to sharpen entity accuracy. All training was performed locally on consumer hardware.

An earlier internal experiment trained a dedicated Chain-of-Thought LoRA on a separate ~1,600-sample narrative-planning dataset. While the model learned the think-block format cleanly, side-by-side A/B testing against Phase 1 alone showed it consistently thinned out prose richness and long-context lore recall — most likely because that dataset was far smaller and less refined than the main 22,000-sample corpus. That phase was not included in this release. The reasoning/thinking behavior available in V15.1 is Gemma 4's own native emergent reasoning, which Phase 1 and Phase 3 were deliberately kept conservative enough (frozen global-attention layers, modest rank, low learning rate, weight decay, LoRA dropout) to preserve rather than overwrite.

Base Model: google/gemma-4-31B-it
Context Adaptations: SWA 1024 → 8192 | RoPE Theta 10,000 → 20,000
Hardware: 4 × RTX 3090 (local workstation)
Framework: Axolotl
Total training time: ~310 hours of wall-clock compute

Phase 1 — SFT & Context Adaptation

LoRA Target Modules: Targeted exclusively SWA Attention (q, k, v, o) and MLP (gate, up, down) layers, deliberately bypassing global attention layers to preserve native architecture capabilities.
Target Regex: lora_target_modules: 'model.language_model.layers.(0|1|2|3|4|6|7|8|9|10|12|13|14|15|16|18|19|20|21|22|24|25|26|27|28|30|31|32|33|34|36|37|38|39|40|42|43|44|45|46|48|49|50|51|52|54|55|56|57|58).(_checkpoint_wrapped_module.)?(self_attn.(q|k|v|o)_proj|mlp.(gate|up|down)_proj)'
LoRA rank: 128
Epochs: 2
Training time: ~290 hours

Phase 3 — Targeted Full Fine-Tune (Entity & Detail Accuracy)

Trained components: Token embeddings, final norm, and sliding-window-layer norms only
Steps: 200
Training time: ~20 hours

Critical: Chat Template & Setup

This model uses the standard Gemma 4 chat template. The structure is complex and requires specific configuration for correct performance. Please read the following sections carefully.

Human-Readable Format:

<bos><|turn>system
<|think|>
{system prompt}<turn|>
<|turn>user
{user message}<turn|>
<|turn>model
<|channel>thought
{reasoning}<channel|>{response}<turn|>

⚠️ Important: Use Chat Completion, Not Text Completion

Strongly recommended: Always use the /v1/chat/completions endpoint. Correct formatting is handled automatically by the server.

Text completion (/v1/completions) is currently broken for impersonations without a custom patch. Sending the context as a user prompt and asking the model to generate the response can produce a malformed turn structure.

Reasoning with text completion and Llama.cpp specific: You must prefill the response with start_reply_with: <|channel>thought + \n (a new line) or reasoning will not trigger (you will get blank responses if you enabled reasoning and did not prefill)

Chat Completion API Configuration (SillyTavern, etc.)

When using an OpenAI-compatible API, you must add custom parameters to control thinking and ensure correct token handling. In frontends like SillyTavern, add the following JSON to the "Custom Parameters" (or extra_body) field in your connection settings:

chat_template_kwargs: {enable_thinking: true, 
preserve_thinking: false}
skip_special_tokens: false

enable_thinking: true: Activates the model's chain-of-thought reasoning. Change to false to disable it for faster, more direct responses.
skip_special_tokens: false: This is critical. It prevents the API from stripping away the special tokens (like <|channel> and <channel|>) that are required for the model's chat template to work correctly.
preserve_thinking: false: Optional, Default: false. newly added to the chat template, It allows reusing previous reasoning turns in

General Notes

The model retains the vision adapter from the base Gemma 4 31B and supports image inputs.
Thinking mode is native to base Gemma 4, not something specifically trained into this fine-tune — the <|think|> token activates it at the architecture level. Training data for V15.1 was written in no-thinking format; reasoning quality when enable_thinking: true reflects Gemma 4's own emergent behavior, preserved by keeping the fine-tune conservative (see Training Process below).
The <|think|> token in the system turn is what activates chain-of-thought reasoning at the architecture level.

Quantized Models

The quantized model files are available for download. Click the button below to view the files.

Download EXL3 Files → Download W8A16-PTQ Files →

Character Card & Lore Book

For the best roleplaying experience, it is highly recommended to use the provided character card and lore book. These files help guide the model's persona and provide rich, in-universe context.

Download Files →

Sampler Presets

For a seamless setup in SillyTavern, you can download pre-configured sampler presets. These are tuned to provide an optimal balance between creativity and narrative coherence for this model.

Simply download the .json file below and import it into SillyTavern's sampler presets menu.

Download SillyTavern Preset (Thinking) → Download SillyTavern Preset (No Thinking) →

For those that dont use silly tavern, Samplers settings are:

Temperature: 1.0

Min P: 0.02

Roleplay Format Guide

For the best results, use this structured format. This helps the AI clearly distinguish between actions, inner thoughts, dialogue, and — when reasoning is enabled — narrative planning.

Actions / Descriptions: *He walked across the room and stared out the window.*
Inner Thoughts: *-I wonder what she's thinking.-*
Dialogue: Alex (Curious): "What do you see out there?"

Roleplay Example

Click the button below to view a full, unedited chatlog demonstrating the model's narrative style and character portrayal.

View Chatlog Example →

Model Description

This is Version 15.1 in the Animus series. Built on google/gemma-4-31B-it, this release features a significantly expanded dataset and revised training composition compared to prior versions.

The Supervised Fine-Tuning (SFT) dataset consists of 122 million tokens across 22,000 samples, structured to balance standard interactions, formatting adherence, and complex narrative paths:

Standard Roleplay & Lore (12,000 samples): Focused on in-character interactions, dialogue, and general world-building.
NSFW Scenarios (4,000 samples): Included to improve narrative flexibility in mature or adult-themed contexts.
Bad Endings (4,000 samples): Designed to handle darker themes, conflict resolution, and complex narrative outcomes.
Instruction Formatting (2,000 samples): Q&A pairs tailored to improve structured task adherence.

This dataset aims to provide steady prose quality, better thematic consistency, and an improved grasp of in-universe lore. A separate Chain-of-Thought dataset was also built and trained as an experimental phase, but was excluded from this release after testing showed it reduced prose richness relative to Phase 1 alone — see the Training Process section below for details. The reasoning/thinking behavior available in V15.1 is Gemma 4's own native emergent reasoning capability, deliberately preserved rather than replaced.

Formatting Note: The model remains highly sensitive to the formatting of the system prompt and the initial turn. If the first assistant message is brief, subsequent responses will generally match that length.

Intended Use & Limitations

Intended Use: The primary purpose of this model is for creative roleplay within the Wings of Fire universe. However, user feedback indicates it is also highly effective for general-purpose roleplaying.
Limitations & Quirks:
- Performance on tasks outside of its training domain (general knowledge, coding, etc.) is not guaranteed and will likely be poor.
- Versatility: While it appears to be only a Wings of Fire tuned model, users have reported it is very capable of performing normal roleplay with other settings and characters.
- Dragon Anatomy Bias (Expected Behavior): The model has a strong prior toward draconic anatomy — wings, claws, talons, scales, snouts, tails — even when roleplaying non-dragon characters. This is intentional and expected: human-anatomy terms (hands, fingers, feet, hair, skin, etc.) were heavily filtered out of the training data to preserve the Wings of Fire setting's integrity. If you want the model to write human or humanoid characters cleanly, provide explicit anatomical cues in the character card or system prompt (e.g. "a human with hands and fingers, no wings or claws"), otherwise expect draconic descriptors to leak through.
- The model may "hallucinate" or generate plausible but non-canonical information, especially when pushed outside the established "what-if" scenarios.
- Content: The training data includes mature and darker themes from the Wings of Fire series, such as conflict, character death, and moral ambiguity. The model is capable of generating content reflecting these themes. As always, it is up to the user what they do with it.
- Formatting: Training data was cleaned to remove narrative artifacts like **scene transitions**. The model should now produce cleaner prose.
- Safety: This model has not undergone additional safety alignment beyond what was included in its base model. Standard responsible AI practices should be followed.

Acknowledgements

Credit to Google for the powerful gemma-4-31B-it model.
Credit to Google for the Gemini Pro model, used in dataset generation.
Credit to Anthropic for sonnet 4.5, used in dataset generation.
Credit to Hangzhou DeepSeek Artificial Intelligence for the deepseek model, used in dataset generation.
Credit to Moonshot AI for the Kimi K2 model, used in dataset generation.

Darkhn-Quants-4/Gemma-4-31B-Animus-V15.1-Swa-GGUF

Gemma-4-31B-Animus-V15.1-Swa-GGUF

⚠️V15.1 Training Process: Make sure to read about the swa change!⚠️