You can deploy this model and start earning money today!

aniketp2009gmail/
phi3-bilora-code-review

Name: phi3-bilora-code-review
Availability: InStock

google/code_x_glue_ct_code_to_text

microsoft/Phi-3-mini-4k-instruct

adapter

Views

No views yet

Phi-3 BiLoRA Code Review

This model is a fine-tuned version of microsoft/Phi-3-mini-4k-instruct using BiLoRA (Dual-Adapter LoRA) for code review tasks, specifically code generation and docstring generation.

Model Details

Model Type: Causal Language Model with multiple LoRA adapters
Base Model: microsoft/Phi-3-mini-4k-instruct
Adapters:
- task_1: Code Generation (fine-tuned on MBPP)
- task_2: Docstring Generation (fine-tuned on CodeXGLUE)
Language(s): Python

Intended Use

This model is intended for code review assistance, including:

Generating Python code from natural language prompts.
Generating descriptive docstrings for existing Python functions.

Training Details

Dataset

Task 1: MBPP (Mostly Basic Python Problems)
Task 2: CodeXGLUE (ct-code-to-text) (Python subset)

BiLoRA Configuration

Rank (r): 4
Alpha: 8
Dropout: 0.1
Target Modules:
- task_1: qkv_proj, o_proj
- task_2: gate_up_proj, down_proj

Hyperparameters

Learning Rate: 2e-4
Batch Size: 1 (Gradient Accumulation Steps: 16)
Epochs: 1
Optimizer: AdamW
LR Scheduler: Linear

Benchmark Results

Evaluation performed on a custom benchmark of 20 samples (10 code gen, 10 docstring gen).

Model	Bug Detection (Pass@1)	Localization (BLEU)	Fix Quality (1-5)	Latency (avg)
BiLoRA (mine)	94.17%	0.0259	3.7/5	33499ms
Phi-3 base	70.0%	0.0536	3.6/5	24561ms
GPT-4 (Groq)	100.0%	0.1255	4.4/5	433ms

Note: Bug Detection is proxied by Code Generation Pass Rate. Localization is proxied by Docstring BLEU score. Fix Quality is an average quality score (1-5).

Example Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(base_model, trust_remote_code=True)
model = PeftModel.from_pretrained(model, "aniketp2009gmail/phi3-bilora-code-review")

tokenizer = AutoTokenizer.from_pretrained("aniketp2009gmail/phi3-bilora-code-review")

# For Code Generation (Task 1)
model.set_adapter("task_1")
prompt = "Generate code: Write a function to find the sum of even numbers in a list\nCode:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

# For Docstring Generation (Task 2)
model.set_adapter("task_2")
prompt = "Generate docstring: def sum_even(lst):\n    return sum(x for x in lst if x % 2 == 0)\nDocstring:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

The model is optimized for Python.
Performance may vary on complex or niche libraries.
Latency is higher than quantized or distilled models.