Views
No views yet
SafeTensors format using transformers 4.41.1. The goal of this reupload is to prevent older models that are still relevant baselines from becoming stale as a result of changes in HuggingFace. Additionally, I may include minor corrections, such as model max length configuration.1from transformers import RobertaTokenizer, RobertaForMaskedLM, pipeline
2
3model = RobertaForMaskedLM.from_pretrained('microsoft/codebert-base-mlm')
4tokenizer = RobertaTokenizer.from_pretrained('microsoft/codebert-base-mlm')
5
6code_example = "if (x is not None) <mask> (x>1)"
7fill_mask = pipeline('fill-mask', model=model, tokenizer=tokenizer)
8
9outputs = fill_mask(code_example)
10print(outputs){'sequence': '<s> if (x is not None) and (x>1)</s>', 'score': 0.6049249172210693, 'token': 8}
{'sequence': '<s> if (x is not None) or (x>1)</s>', 'score': 0.30680200457572937, 'token': 50}
{'sequence': '<s> if (x is not None) if (x>1)</s>', 'score': 0.02133703976869583, 'token': 114}
{'sequence': '<s> if (x is not None) then (x>1)</s>', 'score': 0.018607674166560173, 'token': 172}
{'sequence': '<s> if (x is not None) AND (x>1)</s>', 'score': 0.007619690150022507, 'token': 4248}1@misc{feng2020codebert,
2 title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
3 author={Zhangyin Feng and Daya Guo and Duyu Tang and Nan Duan and Xiaocheng Feng and Ming Gong and Linjun Shou and Bing Qin and Ting Liu and Daxin Jiang and Ming Zhou},
4 year={2020},
5 eprint={2002.08155},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL}
8}