You can deploy this model and start earning money today!

jannik-el/
mikeio-ft-starcoder-lora-a100-gguf

Views

No views yet

mikeio-ft-starcoder-lora-a100-gguf

This repository contains a GGUF quantized version of the finetuned Starcoder model.

Base Model: bigcode/starcoderbase-1b Finetuned on: mikeio documentation Quantization: q4_k_m (or as specified during conversion)

This model can be run locally using Ollama or llama.cpp.

Download the GGUF file:
wget https://huggingface.co/jannik-el/mikeio-ft-starcoder-lora-a100-gguf/resolve/main/merged_starcoder_model.gguf
Create a Modelfile in the same directory:
Modelfile
1FROM ./merged_starcoder_model.gguf 2# Optional: Add a system prompt if your model was finetuned with one 3# SYSTEM You are an assistant that answers questions on MIKE and other software products developed by DHI. 4# Optional: Adjust parameters for inference 5# PARAMETER temperature 0.8 6# PARAMETER top_k 40 7# PARAMETER top_p 0.9 8# PARAMETER num_ctx 2048
Import the model into Ollama:
ollama create my-finetuned-starcoder -f Modelfile
Run the model:
ollama run my-finetuned-starcoder

You can use this GGUF file directly with llama.cpp's main executable:

./main -m merged_starcoder_model.gguf -p "Your prompt here"