Phi-lthy4 has been consistently described as exceptionally unique by all who have tested it, almost devoid of SLOP, and it is now widely regarded as the most unique roleplay model available. It underwent an intensive continued pretraining (CPT) phase, extensive supervised fine-tuning (SFT) on high-quality organic datasets, and leveraged advanced techniques including model merging, parameter pruning, and upscaling.
Interestingly, this distinctiveness was validated in a recent paper: Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification. Among a wide array of models tested, this one stood out as unclassifiable by traditional architecture-based fingerprinting—highlighting the extent of its architectural deviation. This was the result of deep structural modification: not just fine-tuning, but full-layer re-architecture, aggressive parameter pruning, and fusion with unrelated models.
Some things just start on a whim. This is the story of Phi-Lthy4, pretty much:
> yo sicarius can you make phi-4 smarter?
nope. but i can still make it better.
> wdym??
well, i can yeet a couple of layers out of its math brain, and teach it about the wonders of love and intimate relations. maybe. idk if its worth it.
> lol its all synth data in the pretrain. many before you tried.
fine. ill do it.
But... why?
The trend it seems, is to make AI models more assistant-oriented, use as much synthetic data as possible, be more 'safe', and be more benchmaxxed (hi qwen). Sure, this makes great assistants, but sanitized data (like in the Phi model series case) butchers creativity. Not to mention that the previous Phi 3.5 wouldn't even tell you how to kill a process and so on and so forth...
This little side project took about two weeks of on-and-off fine-tuning. After about 1B tokens or so, I lost track of how much I trained it. The idea? A proof of concept of sorts to see if sheer will (and 2xA6000) will be enough to shape a model to any parameter size, behavior or form.
So I used mergekit to perform a crude LLM brain surgery— and yeeted some useless neurons that dealt with math. How do I know that these exact neurons dealt with math? Because ALL of Phi's neurons dealt with math. Success was guaranteed.
Is this the best Phi-4 11.9B RP model in the world? It's quite possible, simply because tuning Phi-4 for RP is a completely stupid idea, both due to its pretraining data, "limited" context size of 16k, and the model's MIT license.
Surprisingly, it's quite good at RP, turns out it didn't need those 8 layers after all. It could probably still solve a basic math question, but I would strongly recommend using a calculator for such tasks.
Why do we want LLMs to do basic math anyway?
Oh, regarding censorship... Let's just say it's... Phi-lthy.
TL;DR
The BEST Phi-4 Roleplay finetune in the world (Not that much of an achievement here, Phi roleplay finetunes can probably be counted on a single hand).
Compact size & fully healed from the brain surgery Only 11.9B parameters. Phi-4 wasn't that hard to run even at 14B, now with even fewer brain cells, your new phone could probably run it easily. (SD8Gen3 and above recommended).
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. Who would have thought?
Smart assistant with low refusals - It kept some of the smarts, and our little Phi-Lthy here will be quite eager to answer your naughty questions.
Quite good at following the character card. Finally, it puts its math brain to some productive tasks. Gooner technology is becoming more popular by the day.
Intended use: Role-Play, Creative Writing, General Tasks.
Censorship level: Medium - Low
5.5 / 10 (10 completely uncensored)
UGI score:
Recommended settings for assistant mode
Full generation settings: Debug Deterministic.
Debug Deterministic_Settings
Full generation settings: min_p.
min_P_Settings
Recommended settings for Roleplay mode
Roleplay settings:.
A good repetition_penalty range is between 1.12 - 1.15, feel free to experiment.
With these settings, each output message should be neatly displayed in 1 - 3 paragraphs, 1 - 2 is the most common. A single paragraph will be output as a response to a simple message ("What was your name again?").
min_P for RP works too but is more likely to put everything under one large paragraph, instead of a neatly formatted short one. Feel free to switch in between.
(Open the image in a new window to better see the full details)
The recommended RP settings will bias towards 1-3 small paragraphs (on some occasions 4-5)
Regarding the format:
It is HIGHLY RECOMMENDED to use the Roleplay \ Adventure format the model was trained on, see the examples below for syntax. It allows for a very fast and easy writing of character cards with minimal amount of tokens. It's a modification of an old-skool CAI style format I call SICAtxt (Simple, Inexpensive Character Attributes plain-text):
SICAtxt for roleplay:
X's Persona: X is a .....
Traits:
Likes:
Dislikes:
Quirks:
Goals:
Dialogue example
<|im_start|>system
You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|>
<|im_start|>User request
{prompt}<|im_end|>
<|im_start|>AI answer
