• Model creator: https://huggingface.co/TeeZee/
• Original model: https://huggingface.co/TeeZee/DarkForest-20B-v1.1/

Original model description:

• merge
• not-for-all-audiences license_name: microsoft-research-license model-index:
• name: DarkForest-20B-v1.2 results:
  • task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
    • type: acc_norm value: 63.57 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
    • type: acc_norm value: 86.42 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
    • type: acc value: 59.77 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
    • type: mc2 value: 56.31 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
    • type: acc value: 77.74 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 24.94 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v1.2 name: Open LLM Leaderboard

---

DarkForest 20B v1.1

Model Details

• To create this model two step procedure was used. First a new 20B model was created using microsoft/Orca-2-13b and KoboldAI/LLaMA2-13B-Erebus-v3 , deatils of the merge in mergekit-config_step1.yml
• then jebcarter/psyonic-cetacean-20B was used to produce the final model, merge config in mergekit-config-step2.yml
• instead of linear merge method used in v1.0, this time DARE TIES method was used for step2
• The resulting model has approximately 20 billion parameters.

Results

• produces SFW nad NSFW content without issues, switches context seamlessly.
• good at following instructions.
• good at tracking multiple characters in one scene.
• very creative, scenarios produced are mature and complicated, model doesn't shy from writing about PTSD, menatal issues or complicated relationships.
• NSFW output is more creative and suprising than typical limaRP output.
• definitely for mature audiences, not only because of vivid NSFW content but also because of overall maturity of stories it produces.
• This is NOT Harry Potter level storytelling.

Open LLM Leaderboard Evaluation Results