Views
No views yet
[!NOTE] This is an unofficial reupload of microsoft/unixcoder-base in theSafeTensorsformat usingtransformers4.40.1. The goal of this reupload is to prevent older models that are still relevant baselines from becoming stale as a result of changes in HuggingFace. Additionally, I may include minor corrections, such as model max length configuration.
UniXcoder has slightly worse BLEU-4 scores on both code summarization and generation tasks. The main reasons may come from two aspects. One is the amount of NL-PL pairs in the pre-training data
We evaluate UniXcoder on five tasks over nine public datasets, including two understanding tasks, two generation tasks and an autoregressive task. To further evaluate the performance of code fragment embeddings, we also propose a new task called zero-shot code-to-code search.
Taking zero-shot code-code search task as an example, after removing contrastive learning, the performance drops from 20.45% to 13.73%.
@misc{https://doi.org/10.48550/arxiv.2203.03850,
doi = {10.48550/ARXIV.2203.03850},
url = {https://arxiv.org/abs/2203.03850},
author = {Guo, Daya and Lu, Shuai and Duan, Nan and Wang, Yanlin and Zhou, Ming and Yin, Jian},
keywords = {Computation and Language (cs.CL), Programming Languages (cs.PL), Software Engineering (cs.SE), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {UniXcoder: Unified Cross-Modal Pre-training for Code 1from transformers import AutoTokenizer, AutoModel
2
3tokenizer = AutoTokenizer.from_pretrained("microsoft/unixcoder-base")
4
5model = AutoModel.from_pretrained("microsoft/unixcoder-base")
6