You can deploy this model and start earning money today!

RichardErkhov/
RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq

Name: RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Phi-3-mini-code-finetune-128k-instruct-v1 - AWQ

Model creator: https://huggingface.co/RDson/
Original model: https://huggingface.co/RDson/Phi-3-mini-code-finetune-128k-instruct-v1/

Original model description:

license: other license_name: phi-3 license_link: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/raw/main/LICENSE datasets:

m-a-p/CodeFeedback-Filtered-Instruction tags:
phi
phi-3
'3'
code

Finetune of microsoft/Phi-3-mini-128k-instruct on m-a-p/CodeFeedback-Filtered-Instruction for ~9-10h using a single 3090 24GB.

Due to limited resources and time, the training was only on half (0.5136) of the epoch.

train_loss: 0.43311

    learning_rate=5e-5,
    lr_scheduler_type="cosine",
    max_length=1024,
    max_prompt_length=512,
    overwrite_output_dir=True,
    beta=0.1,
    gradient_accumulation_steps=8,
    optim="adamw_torch",
    num_train_epochs=1,
    evaluation_strategy="steps",
    eval_steps=0.2,
    logging_steps=1,
    warmup_steps=50,
    fp16=True,
    save_steps=50

RichardErkhov/RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq

Original model description:

RichardErkhov/
RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq