CMG/CMC: CodeT5 (with history)

Details

🔍 For further details, please refer to:

• Paper: https://arxiv.org/abs/2308.07655
• Repository: https://github.com/JetBrains-Research/commit_message_generation

• This model is based on Salesforce/codet5-base checkpoint from 📜 CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation.
• This model was trained with commit diffs as well as WITH commit message history.
• This model was trained on the CommitChronicle dataset introduced in our study.
• Our hyperparameter setting is mostly based on 📜 RACE: Retrieval-augmented Commit Message Generation. The exact values are provided below:

  Hyperparameter	Value
  Encoder context max length	512
  Decoder context max length	512
  Number of training epochs	1
  Batch size	32
  Optimizer	AdamW
  Warmup	Linear
  Number of warmup steps	100
  Peak learning rate	0.00002

Available checkpoints

• Models trained with commit message history:
  • CodeT5: 🤗 JetBrains-Research/cmg-codet5-with-history (this model)
  • CodeReviewer: 🤗 JetBrains-Research/cmg-codereviewer-with-history
  • RACE: 🤗 JetBrains-Research/cmg-race-with-history
• Models trained without commit message history:
  • CodeT5: 🤗 JetBrains-Research/cmg-codet5-without-history
  • CodeReviewer: 🤗 JetBrains-Research/cmg-codereviewer-without-history
  • RACE: 🤗 JetBrains-Research/cmg-race-without-history

Citation