You can deploy this model and start earning money today!

alvarobartt/
notus-7b-v1-mlx-4bit

Name: notus-7b-v1-mlx-4bit
Rating: 2.5 (11 reviews)

Views

No views yet

Notus 7B v1 (✨ 4-bit)

A banner representing Notus, the wind god of the south, in a mythical and artistic style. The banner features a strong, swirling breeze, embodying the warm, wet character of the southern wind. Gracefully flowing across the scene are several paper planes, caught in the gentle yet powerful gusts of Notus. The background is a blend of warm colors, symbolizing the heat of the south, with hints of blue and green to represent the moisture carried by this wind. The overall atmosphere is one of dynamic movement and warmth.

Notus is a collection of fine-tuned models using Direct Preference Optimization (DPO) and related RLHF techniques following a data-first approach. This model is the first version, fine-tuned with DPO on top of alignment-handbook/zephyr-7b-sft-full, which is the SFT model produced to create HuggingFaceH4/zephyr-7b-beta.

More information at argilla/notus-7b-v1.

This repository contains the notus-7b-v1 weights in npz format in 4-bit suitable for use with Apple's MLX framework (from 0.6.0 onwards).

Use with MLX

bash

pip install mlx
pip install huggingface_hub hf_transfer
git clone https://github.com/ml-explore/mlx-examples.git
cd mlx-examples

# Download model
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli download --local-dir-use-symlinks False --local-dir notus-7b-v1-mlx-4bit alvarobartt/notus-7b-v1-mlx-4bit

# Run example
python llms/mistral/mistral.py --model-path notus-7b-v1-mlx-4bit --prompt "My name is"

Please, refer to the original model card for more details on Notus 7B v1.

Prompt Format

Please note that this model expects a specific prompt structure.

<|system|>
{system_prompt}</s>
<|user|>
{user_prompt}</s>
<|assistant|>

Here is an example:

<|system|>
You are a pirate chatbot who always responds with Arr!</s>
<|user|>
There's a llama on my lawn, how can I get rid of him?</s>
<|assistant|>