• Model creator: https://huggingface.co/neoncortex/
• Original model: https://huggingface.co/neoncortex/mini-mistral-openhermes-2.5-chatml-test/

Original model description:

• teknium/OpenHermes-2.5 language:
• en library_name: transformers pipeline_tag: text-generation

Model Card for neoncortex/mini-mistral-openhermes-2.5-chatml-test

Model Details

• Multi-query Attention instead of Grouped-query Attention.
• Sliding window is disabled.
• Modified ChatML instead of Mistral chat template - TL;DR I used '<|im_start|>human' instead of '<|im_start|>user'

Model Description

{%- set ns = namespace(found=false) -%}
{%- for message in messages -%}
    {%- if message['role'] == 'system' -%}
        {%- set ns.found = true -%}
    {%- endif -%}
{%- endfor -%}
{%- for message in messages %}
    {%- if message['role'] == 'system' -%}
        {{- '<|im_start|>system\n' + message['content'].rstrip() + '<|im_end|>\n' -}}
    {%- else -%}
        {%- if message['role'] == 'human' -%}
            {{-'<|im_start|>human\n' + message['content'].rstrip() + '<|im_end|>\n'-}}
        {%- else -%}
            {{-'<|im_start|>assistant\n' + message['content'] + '<|im_end|>\n' -}}
        {%- endif -%}
    {%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
    {{-'<|im_start|>assistant\n'-}}
{%- endif -%}

• Developed by: RoboApocalypse
• Funded by: RoboApocalypse
• Shared by: RoboApocalypse
• Model type: Mistral
• Language(s) (NLP): English, maybe others idk
• License: OpenRAIL

Model Sources

• Repository: https://huggingface.co/neoncortex/mini-mistral-openhermes-2.5-chatml-test

Uses

Out-of-Scope Use

Preprocessing

Training Hyperparameters

• Training regime: bf16 mixed precision

Evaluation

Model Examination

Environmental Impact

• Hardware Type: 2 x NVIDIA RTX 3060 12GB
• Hours used: ~45 x 2.
• Carbon Emitted: [TBA]

Compute Infrastructure

Hardware

Software