aranea-tenebris-120b-v1.0-exl2

• Usable context: ~32768
• Recommended prompt format: Alpaca
• Layers: 137

Quantization

Testing

• Completions for 3rd person narration. No character role was assumed.
• Completions for 1st and 2nd person turn based (out-of-order) rp. A character role was assumed by the model, but narration of minor characters and events was encouraged.

• Consistently short responses.
• Laziness or readily gave up on solving a character problem.
• Overly malleable, where characters could not hold opinions or beliefs.
• Passiveness or an inability to drive the narrative.
• Persistent repeats. Bad merges tend to latch onto and reuse specific keywords.
• Ignoring or missing obvious scenario solutions.
• Impersonating other major characters out of turn during rp tests.
• Faliure to follow a character's description. This criteria is pretty broad, and could include things like character skills, refusals etc.
• Major inconsistencies in scenes or recall. Note - invention of thematically consistant detail was encouraged.

Interesting observations from benchmarking

• 10 layer interleave stride with a 20 layer interleave width consistently outperformed alternative combinations for coherancy.
• 8 layer interleave stride with a 16 layer interleave width consistantly outperformed alternative combinations for creativity whilst remaining reasonably coherant.
• Regular stride intervals are not optimal. In particular offsetting the first or last set of base models offets often improved metrics.
• Goliath-120B is still a good standard for coherancy below 4096 context. A few miqu-1 merges are comparable, but testing found a small amount coherancy could be sacrificed for notable creativity improvements.