You can deploy this model and start earning money today!

RichardErkhov/
TheBlueObserver_-_gemma-2-2b-it-MLX-gguf

Name: TheBlueObserver_-_gemma-2-2b-it-MLX-gguf
Rating: 2.5 (274 reviews)

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

gemma-2-2b-it-MLX - GGUF

Model creator: https://huggingface.co/TheBlueObserver/
Original model: https://huggingface.co/TheBlueObserver/gemma-2-2b-it-MLX/

Name	Quant method	Size
gemma-2-2b-it-MLX.Q2_K.gguf	Q2_K	1.15GB
gemma-2-2b-it-MLX.IQ3_XS.gguf	IQ3_XS	1.22GB
gemma-2-2b-it-MLX.IQ3_S.gguf	IQ3_S	1.27GB
gemma-2-2b-it-MLX.Q3_K_S.gguf	Q3_K_S	1.27GB
gemma-2-2b-it-MLX.IQ3_M.gguf	IQ3_M	1.3GB
gemma-2-2b-it-MLX.Q3_K.gguf	Q3_K	1.36GB
gemma-2-2b-it-MLX.Q3_K_M.gguf	Q3_K_M	1.36GB
gemma-2-2b-it-MLX.Q3_K_L.gguf	Q3_K_L	1.44GB
gemma-2-2b-it-MLX.IQ4_XS.gguf	IQ4_XS	1.47GB
gemma-2-2b-it-MLX.Q4_0.gguf	Q4_0	1.52GB
gemma-2-2b-it-MLX.IQ4_NL.gguf	IQ4_NL	1.53GB
gemma-2-2b-it-MLX.Q4_K_S.gguf	Q4_K_S	1.53GB
gemma-2-2b-it-MLX.Q4_K.gguf	Q4_K	1.59GB
gemma-2-2b-it-MLX.Q4_K_M.gguf	Q4_K_M	1.59GB
gemma-2-2b-it-MLX.Q4_1.gguf	Q4_1	1.64GB
gemma-2-2b-it-MLX.Q5_0.gguf	Q5_0	1.75GB
gemma-2-2b-it-MLX.Q5_K_S.gguf	Q5_K_S	1.75GB
gemma-2-2b-it-MLX.Q5_K.gguf	Q5_K	1.79GB
gemma-2-2b-it-MLX.Q5_K_M.gguf	Q5_K_M	1.79GB
gemma-2-2b-it-MLX.Q5_1.gguf	Q5_1	1.87GB
gemma-2-2b-it-MLX.Q6_K.gguf	Q6_K	2.0GB
gemma-2-2b-it-MLX.Q8_0.gguf	Q8_0	2.59GB

Original model description:

base_model: google/gemma-2-2b-it library_name: transformers license: gemma pipeline_tag: text-generation tags:

conversational
mlx extra_gated_heading: Access Gemma on Hugging Face extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging Face and click below. Requests are processed immediately. extra_gated_button_content: Acknowledge license

TheBlueObserver/gemma-2-2b-it-MLX

The Model TheBlueObserver/gemma-2-2b-it-MLX was converted to MLX format from google/gemma-2-2b-it using mlx-lm version 0.20.2.

Use with mlx

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("TheBlueObserver/gemma-2-2b-it-MLX")

prompt="hello"

if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

RichardErkhov/TheBlueObserver_-_gemma-2-2b-it-MLX-gguf

Original model description:

TheBlueObserver/gemma-2-2b-it-MLX

Use with mlx

RichardErkhov/
TheBlueObserver_-_gemma-2-2b-it-MLX-gguf