Citation
1@misc{Molino2019,
2 author = {Piero Molino and Yaroslav Dudin and Sai Sumanth Miryala},
3 title = {Ludwig: a type-based declarative deep learning toolbox},
4 year = {2019},
5 eprint = {arXiv:1909.07930},
6}
Model Card for Model ID
Model Details
Model Description
This model focuses on fine-tuning the Llama-2 7B large language model for Python code generation. The project leverages Ludwig, an open-source toolkit, and a dataset of 500k Python code samples from Hugging Face. The model applies techniques such as prompt templating, zero-shot inference, and few-shot learning, enhancing the model's performance in generating Python code snippets efficiently.
-
Developed by: Kevin Geejo, Aniket Yadav, Rishab Pandey
-
Model type: Fine-tuned Llama-2 7B for Python code generation
-
Language(s) (NLP): Python (for code generation tasks)
-
License: Not explicitly mentioned, but Llama-2 models are typically governed by Meta AI’s open-source licensing
-
Finetuned from model [optional]: Llama-2 7B (Meta AI, 2023)
Model Sources [optional]
Uses
Direct Use
- Python code generation for software development
- Automation of coding tasks
- Developer productivity enhancement
Downstream Use [optional]
- Code completion, bug fixing, and Python code translation
Out-of-Scope Use
- Non-Python programming tasks
- Generation of sensitive, legal, or medical content
Bias, Risks, and Limitations
- Limited to Python programming tasks
- Dataset biases from Hugging Face's Python Code Dataset
- Environmental impact from computational costs during fine-tuning
Recommendations
Users should be aware of computational efficiency trade-offs and potential limitations in generalizing to new Python tasks.
How to Get Started with the Model
Use the code below to get started with the model:
1# Example setup (simplified)
2import ludwig
3from transformers import AutoModel
4
5model = AutoModel.from_pretrained("llama-2-7b-python")
Training Details
Training Data
- 500k Python code samples sourced from Hugging Face
Training Procedure
- Preprocessing [optional]: Hugging Face Python Code Dataset
- Training regime: Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA)
Speeds, Sizes, Times [optional]
- Not explicitly mentioned in the document
Evaluation
Testing Data, Factors & Metrics
Testing Data
- Derived from Python code datasets on Hugging Face
Factors
- Python code generation tasks
Metrics
- Code correctness and efficiency
Results
- Fine-tuning improved Python code generation performance
Summary
The fine-tuned model showed enhanced proficiency in generating Python code snippets, reflecting its adaptability to specific coding tasks.
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator.
Model Architecture and Objective
- Llama-2 7B model architecture fine-tuned for Python code generation
Compute Infrastructure
Hardware
Software
- Ludwig toolkit and Hugging Face integration
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
- Llama-2: Open-source large language model by Meta AI
- LoRA (Low-Rank Adaptation): Efficient fine-tuning method modifying fewer model parameters
- PEFT: Parameter-efficient fine-tuning technique
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
Kevin Geejo, Aniket Yadav, Rishab Pandey
Model Card Contact
Framework versions
- Llama-2 version: 7B
- Ludwig version: 0.8
- Hugging Face integration: Latest
