You can deploy this model and start earning money today!

ggml-org/
embeddinggemma-300m-qat-q8_0-GGUF

Name: embeddinggemma-300m-qat-q8_0-GGUF
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

embeddinggemma-300m-qat-q8_0 GGUF

Recommended way to run this model:

llama-server -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --embeddings

Then the endpoint can be accessed at http://localhost:8080/embedding, for example using curl:

console

curl --request POST \
    --url http://localhost:8080/embedding \
    --header "Content-Type: application/json" \
    --data '{"input": "Hello embeddings"}' \
    --silent

Alternatively, the llama-embedding command line tool can be used:

llama-embedding -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --verbose-prompt -p "Hello embeddings"

embd_normalize

When a model uses pooling, or the pooling method is specified using --pooling, the normalization can be controlled by the embd_normalize parameter.

The default value is 2 which means that the embeddings are normalized using the Euclidean norm (L2). Other options are:

-1 No normalization
0 Max absolute
1 Taxicab
2 Euclidean/L2
>2 P-Norm

This can be passed in the request body to llama-server, for example:

--data '{"input": "Hello embeddings", "embd_normalize": -1}' \

And for llama-embedding, by passing --embd-normalize <value>, for example:

llama-embedding -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --embd-normalize -1 -p "Hello embeddings"