You can deploy this model and start earning money today!

RichardErkhov/
LimYeri_-_CodeMind-Llama3.1-8B-unsloth-merged-gguf

Name: LimYeri_-_CodeMind-Llama3.1-8B-unsloth-merged-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

CodeMind-Llama3.1-8B-unsloth-merged - GGUF

Model creator: https://huggingface.co/LimYeri/
Original model: https://huggingface.co/LimYeri/CodeMind-Llama3.1-8B-unsloth-merged/

Name	Quant method	Size
CodeMind-Llama3.1-8B-unsloth-merged.Q2_K.gguf	Q2_K	2.96GB
CodeMind-Llama3.1-8B-unsloth-merged.IQ3_XS.gguf	IQ3_XS	3.28GB
CodeMind-Llama3.1-8B-unsloth-merged.IQ3_S.gguf	IQ3_S	1.19GB
CodeMind-Llama3.1-8B-unsloth-merged.Q3_K_S.gguf	Q3_K_S	3.41GB
CodeMind-Llama3.1-8B-unsloth-merged.IQ3_M.gguf	IQ3_M	3.52GB
CodeMind-Llama3.1-8B-unsloth-merged.Q3_K.gguf	Q3_K	3.74GB
CodeMind-Llama3.1-8B-unsloth-merged.Q3_K_M.gguf	Q3_K_M	3.74GB
CodeMind-Llama3.1-8B-unsloth-merged.Q3_K_L.gguf	Q3_K_L	4.03GB
CodeMind-Llama3.1-8B-unsloth-merged.IQ4_XS.gguf	IQ4_XS	0.74GB
CodeMind-Llama3.1-8B-unsloth-merged.Q4_0.gguf	Q4_0	0.74GB
CodeMind-Llama3.1-8B-unsloth-merged.IQ4_NL.gguf	IQ4_NL	4.38GB
CodeMind-Llama3.1-8B-unsloth-merged.Q4_K_S.gguf	Q4_K_S	4.37GB
CodeMind-Llama3.1-8B-unsloth-merged.Q4_K.gguf	Q4_K	4.58GB
CodeMind-Llama3.1-8B-unsloth-merged.Q4_K_M.gguf	Q4_K_M	4.58GB
CodeMind-Llama3.1-8B-unsloth-merged.Q4_1.gguf	Q4_1	4.78GB
CodeMind-Llama3.1-8B-unsloth-merged.Q5_0.gguf	Q5_0	5.21GB
CodeMind-Llama3.1-8B-unsloth-merged.Q5_K_S.gguf	Q5_K_S	5.21GB
CodeMind-Llama3.1-8B-unsloth-merged.Q5_K.gguf	Q5_K	5.34GB
CodeMind-Llama3.1-8B-unsloth-merged.Q5_K_M.gguf	Q5_K_M	5.34GB
CodeMind-Llama3.1-8B-unsloth-merged.Q5_1.gguf	Q5_1	5.65GB
CodeMind-Llama3.1-8B-unsloth-merged.Q6_K.gguf	Q6_K	6.14GB
CodeMind-Llama3.1-8B-unsloth-merged.Q8_0.gguf	Q8_0	7.95GB

Original model description:

base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit language:

en license: mit tags:
text-generation-inference
transformers
unsloth
llama
trl
sft datasets:
LimYeri/LeetCode_Python_Solutions_Data

CodeMind-Llama3.1-8B-unsloth

Codemind Project is a language model developed to assist in solving and learning coding test problems. This model is fine-tuned using posts written by LeetCode users as training data, aiming to provide answers specialized for coding tests.

Model Information

Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
Fine-Tuning: Fine-tuned using the unsloth library based on the unsloth/Meta-Llama-3.1-8B-Instruct model
Fine-Tuning Process: Conducted with reference to the Llama 3.1 Conversational_notebook

Dataset Used

LeetCode Python Solutions Dataset: LeetCode_Python_Solutions_Data

How to Use the Model

This model is accessible through HuggingFace's model hub and can be integrated into applications using the API. It is designed to generate explanations, code snippets, or guides for coding problems or programming-related questions.

python

# 자세한 사항은 demo-Llama3.1.ipynb 확인
from unsloth import FastLanguageModel
from unsloth.chat_templates import get_chat_template
from IPython.display import display, Markdown

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "LimYeri/CodeMind-Llama3.1-8B-unsloth", # YOUR MODEL YOU USED FOR TRAINING
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3.1",
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

messages = [
    {"role": "system", "content": "You are a kind coding test teacher."},
    {"role": "user", "content": "Enter your coding problem or question here."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True, # Must add for generation
    return_tensors = "pt",
).to("cuda")

outputs = model.generate(input_ids = inputs, max_new_tokens = 3000, use_cache = True,
                         temperature = 0.5, min_p = 0.3) # Feel free to adjust the temperature and min_p
text = (tokenizer.batch_decode(outputs))[0].split('assistant<|end_header_id|>\n\n')[1].strip()
display(Markdown(text))

LoRA Configuration

r: 16
lora_alpha: 16
lora_dropout: 0
bias: "none"
use_gradient_checkpointing: "unsloth"

Training Settings

Per Device Train Batch Size: 8
Gradient Accumulation Steps: 2
Warmup Steps: 200
Number of Training Epochs: 5
Learning Rate: 2e-4
fp16: not is_bfloat16_supported()
bf16: is_bfloat16_supported()
Logging Steps: 20
Optimizer: "adamw_8bit"
Weight Decay: 0.01
LR Scheduler Type: "linear"

Evaluation Results Open LLM Leaderboard

Metric	Value
Average	22.17
IFEval	64.9
BBH	24.19
MATH Lvl 5	9.97
GPQA	1.9
MUSR	6.04
MMLU-PRO	26

Fine-Tuning Code

Detailed fine-tuning code and settings can be found in the CodeMind-Extended GitHub repository.

RichardErkhov/LimYeri_-_CodeMind-Llama3.1-8B-unsloth-merged-gguf