This is the adapter of the StarCoder2 model trained using Localized Preference Optimization (LPO) on DiSCo, presented in the paper: "Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferences". The original arXiv preprint can be found at: https://arxiv.org/abs/2506.00419.

To use this model for downstream tasks, you need to merge it with the base model. Specifically, merge it to the model {"bigcode/starcoder2-7b" + StarCoder2-SFT} (where StarCoder2-SFT is an adapter for bigcode/starcoder2-7b).

The associated code repository can be found here: https://github.com/StonyBrookNLP/disco-lpo.

LLM generated code often contains security issues. We address two key challenges in improving secure code generation. First, obtaining high quality training data covering a broad set of security issues is critical. To address this, we introduce a method for distilling a preference dataset of insecure and secure code pairs from frontier LLMs, along with a security reasoning that explains the issues and the fix. The key idea here is to make use of security knowledge sources to devise a systematic prompting strategy that ensures broad coverage. Second, aligning models to secure code requires focusing on localized regions of code. Direct preference optimization methods, like SimPO, are not designed to handle these localized differences and turn out to be ineffective. We address this with a new localized preference optimization algorithm that masks the security related tokens in both the winning (secure) and losing (insecure) responses. To prevent loss in code quality, we also add a regularizer. Evaluations show that both training on our dataset, DiSCo, and the new preference optimization algorithm, LPO, yield substantial reductions in code insecurity while also improving overall code quality. Code and dataset are available at this https URL.

This model is a PEFT adapter. To use it for code generation, you must first merge your chosen base model (e.g., bigcode/starcoder2-7b) with the StarCoder2-SFT adapter. Then, you can apply this StarCoder2-LPO adapter on top of the merged SFT model for inference.

The GitHub repository provides an inference.py script for generation. You can use a command similar to the following, after setting up your environment and merging the SFT adapter:

First, install the required libraries:

Then, run the inference command (example from the GitHub README):

Please refer to the GitHub repository for detailed instructions on merging models and running inference.

Please include the following citation if you are using resources provided in this work: