You can deploy this model and start earning money today!

steamed-potatop/
niska_gemma3_4b_q8

Name: niska_gemma3_4b_q8
Availability: InStock

Views

No views yet

license: mit language:

en
hi base_model:
google/gemma-3-4b-it pipeline_tag: question-answering tags:
legal

NISKA_gemma3_4b_q8 is a 4-billion parameter LLM that, like its predecessor, is fine-tuned for legal tasks. The key difference is that this model is based on Google's Gemma architecture. This means it inherits some of the features of the Gemma family of models, which are known for their performance on a variety of text generation and reasoning tasks.

Using with `llama.cpp`

The instructions for running the NISKA_gemma3_4b_q8.gguf file with llama.cpp are nearly identical to the previous model. The main change is the file name you'll use in the command.

Clone and Build llama.cpp: Follow the same steps as before to clone the llama.cpp repository and build the project with make.
Run the Model: Once built, move your NISKA_gemma3_4b_q8.gguf file into the directory and run it using the main executable. The command will be:

./main -m ./NISKA_gemma3_4b_q8.gguf -n 512 -p "Draft a clause for a confidentiality agreement that covers trade secrets." --color -i

The command breakdown remains the same:
- -m ./NISKA_gemma3_4b_q8.gguf: Specifies the path to your model file.
- -n 512: Sets the maximum number of tokens to generate.
- -p "...": Provides the initial prompt.
- --color: Displays the output in color.
- -i: Runs in interactive mode.

Using with Ollama

The process for using the new model with Ollama is also similar, but you must ensure your Modelfile correctly references the new file name and the base model's architecture (Gemma).

Creating a Modelfile

A Modelfile is the blueprint for creating a custom model in Ollama. It's a simple text file that contains instructions for Ollama, such as which GGUF model to use and what parameters to set. To make it "pretty" and well-documented, you can add comments using the # symbol.

Here is a customized Modelfile for NISKA_gemma3_4b_q8:

# This Modelfile defines the NISKA Legal Assistant, a specialized LLM for legal text generation.
# It is based on the Gemma architecture and fine-tuned for a legal corpus.

FROM ./NISKA_gemma3_4b_q8.gguf

# Optional: Set default parameters for a more controlled output.
# Temperature controls randomness. Lower values are more deterministic.
PARAMETER temperature 0.7
# Top_k limits the number of tokens to consider when sampling.
PARAMETER top_k 40
# Top_p is the cumulative probability for sampling.
PARAMETER top_p 0.9

# Define the system prompt, which gives the model its "persona".
SYSTEM """
You are NISKA, a specialized legal assistant based on the Gemma architecture.
Your purpose is to provide information and draft text based on legal concepts.
You must not provide professional legal advice. All information is for informational and educational purposes only.
"""

Creating the Model in Ollama

Save the Modelfile: Save the text above in a file named Modelfile (with no extension) in the same directory as your NISKA_gemma3_4b_q8.gguf file.
Run the ollama create command: Open your terminal, navigate to the directory, and run the following command to create the model in Ollama.

ollama create niska-gemma-legal -f Modelfile
- niska-gemma-legal: This is the name you are assigning to your model within Ollama.
- -f Modelfile: This tells Ollama to use the instructions from the Modelfile you created.
Run the Model: Once the creation process is complete, you can start an interactive chat session with your new model.

ollama run niska-gemma-legal

or just fetch to ollama from "hf.co"

run this in the terminal (make sure ollama is installed and running)

ollama run hf.co/steamed-potatop/niska_gemma3_4b_q8

steamed-potatop/niska_gemma3_4b_q8

Using with llama.cpp