You can deploy this model and start earning money today!

suka-nlp/
Llama3_finetune

Name: Llama3_finetune
Availability: InStock

adapter-transformers

code

mlabonne/Evol-Instruct-Python-26k

Views

No views yet

Model Details

Model Description

Developed by: Luthfantry Zamzam Muhammad
Model type: large language model for code generation
Language(s) (NLP): English
Finetuned from model: llama3

Model Sources

Repository: https://github.com/unslothai/unsloth
Developed by: unsloth

Model parameter

r = 16,
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj","gate_proj", "up_proj", "down_proj",],
lora_alpha = 16,
lora_dropout = 0,
bias = "none",
use_gradient_checkpointing = "unsloth",
random_state = 3407,
use_rslora = False,
loftq_config = None,

Usage and limitations

This model is used to generate code based on commands given by the user. it should be noted that this model can generate many languages because it takes the initial model from llama2. However, after finetuning it is better at generating python code, because currently it is only trained with python code datasets.

How to Get Started with the Model

use link below to use model https://colab.research.google.com/drive/1P8hQ14Kz5kCMznk8Mg-92sgVN49huOEK?authuser=4#scrollTo=2eSvM9zX_2d3

Training Data

https://huggingface.co/datasets/mlabonne/Evol-Instruct-Python-26k

Training Hyperparameters

Warmup_step: 5
lr_scheduler_type: linear
Learning Rate: 0.0002
Batch Size: 2
Weigh_decay: 0.001
Epoch: 60
Optimizer: adamw_8bit

Testing Data

https://huggingface.co/datasets/google-research-datasets/mbpp/viewer/full

Testing Document

https://docs.google.com/spreadsheets/d/1hr8R4nixQsDC5cGGENTOLUW1jPCS_lltVRIeOzenBvA/edit?usp=sharing

Results

Berfore finetune

Accurary : 32%
Consistensy : 0%

After fine tune

Accuracy : 59%
Consistency : 100%

suka-nlp/Llama3_finetune