You can deploy this model and start earning money today!

mayankpuvvala/
peft_lora_t5_merged_model_pytorch_issues

Name: peft_lora_t5_merged_model_pytorch_issues
Availability: InStock

text-generation-inference

endpoints_compatible

Views

No views yet

Model Card for `mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues`

This model is a fine-tuned T5 encoder-decoder designed to generate GitHub issue bodies based on issue titles, specifically for the PyTorch repository. It utilizes Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA) to achieve efficient training and inference.

Model Details

Model Description

Developed by: Mayank Puvvala
Model type: Text-to-Text Generation
Language(s): English
License: MIT
Fine-tuned from model: t5-small

Model Sources

Repository: GitHub
Demo: Hugging Face Space

Uses

Direct Use

Input: GitHub issue title (e.g., "Memory leak when using DataLoader with num_workers > 0")
Output: Generated issue body describing the problem in detail.

Out-of-Scope Use

Not suitable for generating issue bodies for repositories other than PyTorch without further fine-tuning.

Bias, Risks, and Limitations

The model is trained on PyTorch issues and may not generalize well to other domains.
Generated content should be reviewed for accuracy and relevance.

Recommendations

Use the model as an assistant tool, with human oversight for the final content.

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

input_text = "Memory leak when using DataLoader with num_workers > 0"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Custom dataset of PyTorch GitHub issues, comprising titles and corresponding bodies.

Training Procedure

Fine-tuned using PEFT with LoRA for 3 epochs.

Training Hyperparameters

Epochs: 3
Batch size: 8

Evaluation Metrics

ROUGE Precision: 53.12%
ROUGE F1 Score: 49.8%

mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues

Model Card for mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues