You can deploy this model and start earning money today!

Pradnya27/
codegpt-finetuned-code-generation

Name: codegpt-finetuned-code-generation
Availability: InStock

safetensors

gpt2

Views

No views yet

CodeGPT Fine-tuned for Code Generation

Model Description

This model is a fine-tuned version of microsoft/CodeGPT-small-py trained on coding problems and solutions for code generation tasks.

Training Details

Base Model: microsoft/CodeGPT-small-py (124M parameters)
Dataset: Rabinovich/Code-Generation-LLM-LoRA (500 examples)
Epochs: 2
Learning Rate: 5e-5
Batch Size: 4
Hardware: CPU

Training Results

Step	Training Loss
25	4.4322
50	3.4648
100	3.1430
150	2.7050
200	2.7491
250	2.7126

Loss improved from 4.43 → 2.71 (39% reduction)

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Pradnya27/codegpt-finetuned-code-generation")
tokenizer = AutoTokenizer.from_pretrained("Pradnya27/codegpt-finetuned-code-generation")

prompt = "Generate code: Write a function to check if a number is prime"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(inputs["input_ids"], max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Trained on a small subset (500 examples) — larger training would improve results
Works best with competitive programming style problems
Output quality improves with more specific prompts

Future Work

Train on full dataset (34,727 examples)
Experiment with LoRA fine-tuning
Evaluate on HumanEval benchmark