You can deploy this model and start earning money today!

gghfez/
Writer-Large-2411-v2.1-AWQ

Name: Writer-Large-2411-v2.1-AWQ
Availability: InStock

safetensors

mistral

gghfez/Writer-Large-2411-v2.1

Views

No views yet

Writer-Large-2411-v2.1

AWQ-Quant of gghfez/Writer-Large-2411-v2.1

Creative-Writing Control-Vectors available here: gghfez/Writer-Large-2411-v2.1-control-vectors

Overview

This model is built on Mistral-Large-Instruct-2411 and optimized for creative writing purposes. The base model excels at following instructions and handling details in long context when using the new prompt template.

Key Improvements

Reduced positivity bias
Reduced AI tropes and repetitive language patterns in story generation
Enhanced performance with longer context stories (multiple chapters) and roleplay sessions
Improved steering capabilities for roleplay via [OOC] instructions
Better handling of "group chat" scenarios

Usage

Prompt Template

The model requires a system prompt in the Mistral-V7 format. If you omit [SYSTEM_PROMPT] [/SYSTEM_PROMPT], the model:

May not follow instructions properly at short contexts
Can become repetitive at longer contexts

Example:

[SYSTEM_PROMPT]You are an award winning writer. Assist the user.[/SYSTEM_PROMPT][INST] Write the opening chapter of ... [/INST]

SillyTavern Integration

Story String:

python

[SYSTEM_PROMPT] {{#if system}}{{system}}[/SYSTEM_PROMPT] [INST]
{{/if}}{{#if wiBefore}}{{wiBefore}}
{{/if}}{{#if description}}{{description}}
{{/if}}{{#if personality}}{{personality}}
{{/if}}{{#if scenario}}{{scenario}}
{{/if}}{{#if wiAfter}}{{wiAfter}}
{{/if}}{{#if persona}}{{persona}}
{{/if}}{{trim}}[/INST] Understood.</s>

For response steering, use [OOC] commands, e.g.:

[OOC] Have them interrupted by a loud explosion in a nearby factory
[OOC] Have her refuse to sell it and suggest another merchant instead

Technical Details

Training

QLoRA training at 32768 context
Merged with gghfez/Mistral-Large-Instruct-2411 at bf16
jukofyork/Creative writing control vectors were applied during synthetic dataset generation
Includes standard assistant instruct data for long-context stability
Note: Performance on code tasks may be reduced compared to base model
Note: No attempt was made to remove 'Name-Slop', so you'll still encounter Lily and Elara if you don't specify character names

Context Length

Base model: 131,072 tokens
Training range: 1024-32728 tokens
Training context window: 32768 tokens

Testing Environments

Tested with exllamav2 4.5bpw on:

gghfez/Writer-Large-2411-v2.1-AWQ