...I should explain what the 2 oiiaioiiai models are. And why I'm keeping this.
The inital idea for oiiaioiiai is the giant "final"/"farewell" model for me on L3.3. Basically, take all the best and greatest as of this month, 04/25. and rock smashing into a huge model.
The server I use to merge models only has around 200GB of ram. Initally I tried doing the whole suite of ~28 odd so models... But as you can already tell, this would have crashed the VM. So instead, I split it into 2 models.
What currently A is actually the second iteration of the A part. Why?
Well if anyone have seen KaraKaraWarehouse/L3.3-Joubutsu2000... That model is a living example of why it's risky merging in so many models. Even at low temperatures, it still has no concept of a "sentence".
Hence the title was renamed from "A" to "Joubutsu".
(If you really want to learn what it is, I highly suggest looking up on dic.nicovideo.jp)
After testing "Joubutsu" (formerly oiiai-A), I decided to change things up a bit for B. dare_ties became ties only. Since I suspected the poor model became... like that... because of the merge method was different.
Then I tested B, and it was... usable. At the time, I'm also was looking for a model to do MTL with.
I'm kinda blown away by how much I like it when it comes to doing Japanese to English translations. Which I felt my previous models didn't do super well.
(Perhaps... It could be due to Bigger body or Asobi or any other model, but your guess is good as mine...)
More so, this model also is one of the first models I managed to merge in AISG's SEA Lion models. (#SupportLocal and I do geniunely mean that)
So the theory should hold that it should be able to do Burmese, Chinese, English, Filipino, Indonesia, Javanese, Khmer, Lao, Malay, Sundanese, Tamil, Thai, Vietnamese better than it's counterparts.
Anyway, A2 (The iteration that is on HF and labeled as A) faired a bit better but still write like a schizopheric after some quick testing.
Model Vibes
I'm writing vibes from my testing. Trying to be a bit more transparent on some of my models since I'm sure some that use on featherless would appreciate the added transparency.
Plus, it's aspirations to new model mergers on what to look out for their own merges.
For my usage I actually do not RP. I love to theory-craft and write scenarios and game events (possibly due to my background but I digress). So take it from that POV:
Tends to write wikipedia like content when asking for idea crafting. It doesn't like giving point forms.
Japanese -> English Translation (YMMV), is better than MagicalGirl series and is the reason why I'm keeping this model around.
Has a bit of a "Weeb" feel to it(?). It tends to stretch out groans and moans like you might see in a Visual Novel.
This model like to be more "direct" and pays more attention to the prompt.
It is able to do dialogue generation (Likely due to wayfarer playing a part in it). But I prefer CursedMagicalGirl or MagicalGirl-2 over this.
Model should be able to do furry elements well, but I haven't tested it.
Base model should make it uncensored. But I think it might be a tad too horny.
Prompting
ChatML seems to work which is what I typically use.
My standard sampler settings are as follows (finally and yes, it's more or less standarized)
Temperature: 1.4/1.2
Min P: 0.03
How 2 Model Merge
I'm considering writing an article for this so I'm jotting my notes down in public.
On Selection...
In general and "Fun fact". I do not have a rhyme or reason for merging. The reason why I call model merging "Rock smashing" is because it's like one sometimes. You bang 2 different rocks and you get another different rock. You could do all the fancy science nonsense but I personally not a fan of those.
Though I think there should be a few points to be made:
You should probably pre-test select models and read their model description.
Benchmarks can only get you that far before you realize that "there is no perfect benchmark in the world, and any and all benchmarks can be cheated."
I'll add more when I can think of them.
On Testing models
Due to 2, there is also no standard way to test models. Of course you should be able to tell if the model is behaving like a schizo.
Test with what you have. Your own chat logs is a useful way to tell what is good and what is bad
Write down what you observe compared to other models. (See section on model vibes above).
In a way, they help inform you what your next model can do better or you want out of a model.
(i.e. I'm currently considering merging magicalgirl with this model and see if I can get the best out of both worlds)
The following YAML configuration was used to produce this model:
yaml
1##############################################################################2# The benefit of L3 models is that all subversions are mergable in some way.3# So we can create something **REALLY REALLY REALLY** Stupid like this.4##############################################################################5# PLEASE DO NOT FOLLOW.6# This will probably show up on the hf repo. Hi there!7##############################################################################8# - KaraKaraWitch.9# P.S. 3e7aWKeGHFE (15/04/25)10##############################################################################1112models:13-model: Black-Ink-Guild/Pernicious_Prophecy_70B
14parameters:15density:0.812916weight:0.337817# De-alignment18-model: PKU-Alignment/alpaca-70b-reproduced-llama-319parameters:20density:0.790921weight:0.67222# Text Adventure23-model: LatitudeGames/Wayfarer-Large-70B-Llama-3.324parameters:25density:0.543526weight:0.761927-model: KaraKaraWitch/Llama-3.3-Amakuro
28parameters:29density:0.3730weight:0.35931-model: ReadyArt/Forgotten-Safeword-70B-v5.0
32parameters:33density:0.3734weight:0.35935-model: Undi95/Sushi-v1.4
36parameters:37density:0.62338weight:0.78939-model: sophosympatheia/Nova-Tempus-70B-v0.1
40parameters:41density:0.34442weight:0.638243-model: flammenai/Mahou-1.5-llama3.1-70B
44parameters:45density:0.5649046weight:0.459747# Changelog: [ADDED] furries.48-model: Mawdistical/Draconic-Tease-70B
49parameters:50density:0.470651weight:0.369752# R1 causes a lot of alignment. So we avoid it.53-model: Steelskull/L3.3-Electra-R1-70b
54parameters:55density:0.169256weight:0.169257# Blue hair, blue tie... Hiding in your wiifii58# - model: sophosympatheia/Midnight-Miqu-70B-v1.059# parameters:60# density: 0.470661# weight: 0.369762# OpenBioLLM does not use safetensors in the repo. Custom safetensors version.63-model: OpenBioLLM
64parameters:65density:0.26766weight:0.181767-model: allura-org/Bigger-Body-70b
68parameters:69density:0.675170weight:0.372271-model: nbeerbower/Llama3-Asobi-70B
72parameters:73density:0.711374weight:0.470675# ...Reminds that anytime that I try and merge in SEALION models76# it ends up overpowering other models. So I'm setting it *really* low.77-model: aisingapore/Llama-SEA-LION-v3-70B-IT
78parameters:79density:0.052780weight:0.1193818283merge_method: ties
84base_model: ReadyArt/The-Omega-Directive-L-70B-v1.0
85parameters:86select_topk:0.5087dtype: bfloat16
88
