Another experimental release. Mistral Small 3.2 24B upscaled by 18 layers to create a 33.6B model. This model then went through pretraining, SFT & DPO.

Can't guarantee the Mistral 3.2 repetition issues are fixed, but this model seems to be less repetitive than my previous attempt.

This is an uncensored creative model intended to excel at character driven RP / ERP where characters are portrayed creatively and proactively.

SillyTavern Settings

Recommended Roleplay Format

> Actions: In plaintext

> Dialogue: "In quotes"

> Thoughts: *In asterisks*

Recommended Samplers

> Temp: 0.6

> MinP: 0.03 - 0.05

> TopP: 0.95 - 1.0

> Dry: 0.8, 1.75, 4

Instruct

Mistral v7 Tekken

Quantizations

GGUF

Static (mradermacher)

iMatrix (mradermacher)

EXL3

3bpw

4bpw

5bpw

6bpw

Creation Process

Creation process: Upscale > Pretrain > SFT > DPO

All training was qlora (including pretrain).

Pretrained on 177MB of data. Dataset consisteted mostly of Light Novels, NSFW stories, SFW stories and filled out with general corpus text from Huggingface FineWeb-2 dataset.

The model then went through SFT using a dataset of approx 3.6 million tokens, 700 RP conversations, 1000 creative writing / instruct samples and about 100 summaries. The bulk of this data has been made public.

Finally, DPO was used to make the model more consistent.

base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
merge_method: passthrough
dtype: bfloat16
slices:
  - sources:
      - model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
        layer_range: [0, 29]
  - sources:
      - model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
        layer_range: [10, 39]

Not optimized for cost / performance efficiency, YMMV.

SFT 1*H100

# ====================
# MODEL CONFIGURATION
# ====================
base_model: ./Upscale_Mistral-PT/merged
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
chat_template: mistral_v7_tekken
[object Object]
# ====================
# DATASET CONFIGURATION
# ====================
datasets:
  - path: ./dataset.jsonl
    type: chat_template
    split: train
    chat_template_strategy: tokenizer
    field_messages: messages
    message_property_mappings:
      role: role
      content: content
    roles:
      user: ["user"]
      assistant: ["assistant"]
      system: ["system"]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object]
[object Object][object Object]

zerofata/MS3.2-PaintedFantasy-Visage-33B_exl3_4bpw

PAINTED FANTASY VISAGE

Overview