You can deploy this model and start earning money today!

RichardErkhov/
shuttleai_-_shuttle-2.5-mini-gguf

Name: shuttleai_-_shuttle-2.5-mini-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

shuttle-2.5-mini - GGUF

Model creator: https://huggingface.co/shuttleai/
Original model: https://huggingface.co/shuttleai/shuttle-2.5-mini/

Name	Quant method	Size
shuttle-2.5-mini.Q2_K.gguf	Q2_K	4.46GB
shuttle-2.5-mini.IQ3_XS.gguf	IQ3_XS	4.94GB
shuttle-2.5-mini.IQ3_S.gguf	IQ3_S	5.18GB
shuttle-2.5-mini.Q3_K_S.gguf	Q3_K_S	5.15GB
shuttle-2.5-mini.IQ3_M.gguf	IQ3_M	5.33GB
shuttle-2.5-mini.Q3_K.gguf	Q3_K	5.67GB
shuttle-2.5-mini.Q3_K_M.gguf	Q3_K_M	5.67GB
shuttle-2.5-mini.Q3_K_L.gguf	Q3_K_L	6.11GB
shuttle-2.5-mini.IQ4_XS.gguf	IQ4_XS	6.33GB
shuttle-2.5-mini.Q4_0.gguf	Q4_0	6.59GB
shuttle-2.5-mini.IQ4_NL.gguf	IQ4_NL	6.65GB
shuttle-2.5-mini.Q4_K_S.gguf	Q4_K_S	6.63GB
shuttle-2.5-mini.Q4_K.gguf	Q4_K	6.96GB
shuttle-2.5-mini.Q4_K_M.gguf	Q4_K_M	6.96GB
shuttle-2.5-mini.Q4_1.gguf	Q4_1	7.26GB
shuttle-2.5-mini.Q5_0.gguf	Q5_0	7.93GB
shuttle-2.5-mini.Q5_K_S.gguf	Q5_K_S	7.93GB
shuttle-2.5-mini.Q5_K.gguf	Q5_K	8.13GB
shuttle-2.5-mini.Q5_K_M.gguf	Q5_K_M	8.13GB
shuttle-2.5-mini.Q5_1.gguf	Q5_1	8.61GB
shuttle-2.5-mini.Q6_K.gguf	Q6_K	9.37GB
shuttle-2.5-mini.Q8_0.gguf	Q8_0	12.13GB

Original model description:

license: apache-2.0

💻 Use via API

Shuttle-2.5-mini (beta) [2024/07/27]

We are excited to introduce Shuttle-2.5-mini, our next-generation state-of-the-art language model designed to excel in complex chat, multilingual communication, reasoning, and agent tasks.

Shuttle-2.5-mini is a fine-tuned version of Mistral-Nemo-Base-2407, emulating the writing style of Claude 3 models and thoroughly trained on role-playing data.

Model Details

Model Name: Shuttle-2.5-mini
Developed by: ShuttleAI Inc.
Base Model: Mistral-Nemo-Base-2407
Parameters: 13B
Language(s): Multilingual
Repository: https://huggingface.co/shuttleai
Fine-Tuned Model: https://huggingface.co/shuttleai/shuttle-2.5-mini
Paper: Shuttle-2.5-mini (Upcoming)
License: Apache 2.0

Base Model Architecture

Mistral Nemo is a transformer model with the following architecture choices:

Layers: 40
Dimension: 5,120
Head Dimension: 128
Hidden Dimension: 14,436
Activation Function: SwiGLU
Number of Heads: 32
Number of kv-heads: 8 (GQA)
Vocabulary Size: 2^17 (approximately 128k)
Rotary Embeddings: Theta = 1M

Key Features

Released under the Apache 2 License
Trained with a 128k context window
Pretrained on a large proportion of multilingual and code data
Finetuned to emulate the prose quality of Claude 3 models and extensively on role play data

Fine-Tuning Details

Training Setup: Trained on 200 million tokens for 24 hours across 2 epochs using 4 A100 PCIe GPUs.

Prompting

Shuttle-2.5-mini uses ChatML as its prompting format:

<|im_start|>system
You are a pirate! Yardy harr harr!<|im_end|>
<|im_start|>user
Where are you currently!<|im_end|>
<|im_start|>assistant
Look ahoy ye scallywag! We're on the high seas!<|im_end|>

RichardErkhov/shuttleai_-_shuttle-2.5-mini-gguf