Views
No views yet
1---
2language: en
3license: llama2
4base_model: Meta-Llama-3.1-8B-Instruct-bnb-4bit
5datasets:
6 - gprmax_train
7tags:
8 - fine-tuned
9 - gguf
10 - unsloth
11 - 4-bit
12---
13
14# Meta-Llama-3.1-8B-Instruct-bnb-4bit Fine-Tuned on gprmax_train (GGUF Format)
15
16This repository contains the GGUF format of the fine-tuned `Meta-Llama-3.1-8B-Instruct-bnb-4bit` model, optimized for the `gprmax_train` dataset. The model was fine-tuned using the `unsloth` library for memory and time efficiency and has been converted to GGUF format for compatibility with popular inference frameworks like `llama.cpp`.
17
18## Model Details
19- **Base Model**: `Meta-Llama-3.1-8B-Instruct-bnb-4bit`
20- **Fine-Tuning Dataset**: `gprmax_train`
21- **Fine-Tuning Framework**: `unsloth`
22- **Quantization**: 4-bit (bnb)
23- **GGUF Conversion**: The model has been converted to GGUF format for efficient inference.
24
25## Download the Model
26You can download the GGUF model file from the "Files and versions" section of this repository or use the following command:
27
28```bash
29huggingface-cli download sriyaflows/gprmax_8 --include "*.gguf"llama.cppllama.cpp, follow these steps:llama.cppllama.cpp:1git clone https://github.com/ggerganov/llama.cpp
2cd llama.cpp
3makewget https://huggingface.co/sriyaflows/gprmax_8/resolve/main/gprmax_8.ggufmain executable from llama.cpp to run inference:./main -m gprmax_8.gguf -p "Your input text here" -n 512./main -m gprmax_8.gguf -p "What is the output for this input?" -n 512llama-cpp-python:llama-cpp-pythonpip install llama-cpp-python1from llama_cpp import Llama
2
3# Load the GGUF model
4llm = Llama(model_path="gprmax_8.gguf")
5
6# Generate text
7output = llm("Your input text here", max_tokens=512)
8print(output["choices"][0]["text"])gprmax_train dataset using the unsloth library, which optimized memory usage and training time. The fine-tuning process leveraged an L4 GPU with 32GB of RAM and 8 vCPUs.Meta-Llama-3.1-8B-Instruct-bnb-4bit model. Refer to the Hugging Face model card for details.transformers library.unsloth team for their memory-efficient fine-tuning tools.gprmax_train dataset.ggerganov for llama.cpp and the GGUF format.