An intermediate release. The best Nightbloom so far, but not the finished article.
Nightbloom is a 36B creative writing and roleplaying model based on ByteDance's SeedOSS, trained through repeated cycles of supervised fine-tuning and ORPO alignment on a hand-curated dataset of literature, frontier model generations, original writing, and SillyTavern conversation logs.
What It Does
Roleplaying: Excels at long-context character work. Stable through 80+ turn sessions where frontier models lose the thread. Good at finding signals in sparse conversational data.
Creative writing / voice imitation: Trained heavily on its creator's writing. It can produce credible blog posts, essays, and prose in a specific literary voice — because that voice is a significant portion of the training data.
Character work: Strong at "larger than life" characters — high audacity, willing to take risks that frontier models won't. Better at causing trouble than tiptoeing around it.
What It Doesn't Do Well (Yet)
First-turn creativity: Strong in sustained conversation, weaker on cold starts. This is a known signal from long-conversation training bias — the model learned to find its footing from context, not from nothing.
Temperature sensitivity: Works well between 0.6–0.87. Outside that range, quality degrades. This is likely over-ORPO narrowing; the next training round will include an SFT pass to widen the operating envelope.
Verisimilitude: Occasionally generates fantastic prose followed by an excess creative leap — a character seeing through walls, an invented detail that breaks immersion. This is improving with each round but not yet solved.
Training History
Nightbloom has been trained through an iterative self-curation process:
SFT on a growing dataset (now ~150MB SFT, ~100MB ORPO) of literature, frontier outputs, and original writing
ORPO rounds using chosen/rejected pairs built from the model's own outputs — the model generates, outputs are graded by hand, and the good/bad pairs are fed back as training signal
User-side ORPO where SillyTavern conversation logs are converted to training pairs: the human's actual messages become "chosen" and model-generated user responses become "rejected," teaching the model how a specific person writes
SFT→ORPO cycling — multiple rounds of SFT to teach new capabilities followed by ORPO to sharpen judgment
v1g specifically is the result of:
SFT on 5M tokens extracted from prior ORPO data (v1de)
ORPO on causality/emotion pairs (~1001 train, 125 valid, 126 test)
Extended ORPO run with gradient accumulation (batch 4, grad_accum 4, lr 6e-6, beta 0.37)
Checkpoint at iteration 660, selected after overcook detected at 1140
All tunes are full-depth LoRA on mlx-lm. ~100+ hours of total training time across all versions.
Datasets
Literature and creative writing (published and generated)
Frontier model outputs (curated for quality)
Original writing by the creator (>50% of the dataset by volume)
SillyTavern conversation logs (processed via custom tooling into both SFT and ORPO formats)
Purpose-built ORPO pairs for creativity, literary style, audacity, causality, and emotional reading
Settings
Temperature: 0.6–0.87. The model is temperature-sensitive; ideal generation is in the 0.65–0.75 range. At higher temperatures it spirals; lower temperatures produce flat output.
System prompt: Not required, but recommended:
You are roleplaying with the user.
(any custom details)
## Roleplaying Details
The training data provides character information in the first user message rather than via system prompt, but the model adapts well to either format.
Censorship: None detected. The base model (Blossom/tainted-heresy) has a very low refusal rate.
Why Release an Intermediate?
Because it's already useful enough that its creator uses it preferentially over frontier models for certain tasks — specifically, long RP sessions and writing in a specific voice. The flaws are known and documented. The next training round (prose-focused SFT to widen the temperature envelope and improve first-turn creativity) will address the remaining issues.
The goal is a v1 release where the model's flaws are minor enough that they don't detract from its strengths. v1g is probably 70% of the way there.
Layer Behavior (For Merge Experimenters)
Approximate layer structure of the SeedOSS 36B base:
Layers
Function
0–24
Processing/syntax — don't touch
24–32
Upper reasoning — duplication causes paranoia
32–48
Prosaic/definition
48–60
Adjective/adverb/phrase choice
60–64
Don't touch or it stops working
Flaws
Poor awareness of gendered details (clothing, anatomy) — inherited from source model
