You can deploy this model and start earning money today!

dnx04/
cl-amp

Views

No views yet

CL-AMP

CL-AMP is a Python research prototype for function-level vulnerability detection. It uses vulnerability or safety descriptions as training supervision, but performs inference with source code only.

The current implementation is the model2 variant: a UniXCoder-based encoder with sliding-window code encoding, learned attention pooling, a code-only simulator, and an MLP classifier.

Current Status

This repository is not a clean package release. It is a working research workspace with model code, datasets, checkpoints, logs, thesis notes, and archived experiments.

Important current facts:

Main CL-AMP CLI: clamp.py
Main model: model.py
Shared metrics and inference helpers: util.py
Baselines: baselines/
Dataset conversion workflow: convert Parquet to Arrow IPC before training or evaluation.
Existing checkpoint: models/clamp_r2vul.pt
The current tree does not contain main.py, train.sh, adapt.sh, eval.sh, primevul_eval.py, or root-level baseline runners.

Repository Layout

text

.
├── clamp.py                 # CL-AMP train/adapt/eval Typer CLI
├── model.py                 # CL-AMP model2 architecture and losses
├── util.py                  # inference, threshold tuning, metrics
├── pyproject.toml           # uv/Python dependencies
├── uv.lock                  # locked uv dependency resolution
├── conda.yml                # lightweight conda dependency file
├── ablation.md              # ablation command notes
├── datasets/
│   ├── dataset_schema.json  # normalized dataset schema
│   ├── parquet_to_arrow.py  # Parquet -> Arrow IPC converter
│   ├── llm_augment.py       # OpenAI-compatible description generation
│   └── paired_to_single.py  # paired dataset expansion utility
├── baselines/
│   ├── linevul_runner.py
│   ├── regvd_runner.py
│   ├── clever_runner.py
│   ├── gemma_prompt_runner.py
│   └── *_model.py
├── docs/
│   ├── clever.md
│   ├── clever.pdf
│   └── fpt-ai.md
├── logs/                    # logged experiment outputs
├── models/                  # local checkpoints
└── archives/                # older runners, models, caches, and logs

The VS Code workspace file also references a sibling thesis folder: ../thesis.

Environment

Use the checked-in virtual environment when available:

bash

./.venv/bin/python --version
./.venv/bin/python -c "import torch, transformers, pyarrow, pandas, sklearn, typer; print('ok')"

Or recreate/sync dependencies with uv:

bash

uv sync
uv run python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

The project targets GPU training. pyproject.toml points uv at the PyTorch CUDA 13.0 wheel index:

toml

[tool.uv]
index-url = "https://pypi.org/simple"
extra-index-url = ["https://download.pytorch.org/whl/cu130"]
index-strategy = "unsafe-best-match"

On GPU servers where PyTorch/CUDA is already managed by conda, update the active environment with only the extra project dependencies:

bash

conda env update -n main -f conda.yml --verbose
conda activate main
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

Dataset Format

The normalized schema is documented in datasets/dataset_schema.json.

Column	Type	Required	Description
`code`	string	yes	Source function or code snippet.
`label`	int	yes	`0` = safe, `1` = vulnerable.
`cwe_id`	list/string-like	no	CWE identifier(s), empty when unavailable.
`description`	string	no	Vulnerability/safety reasoning used during training.
`lang`	string	no	Language tag used for breakdowns when present.

Training and evaluation loaders expect Arrow IPC files. Many datasets in this workspace are stored as Parquet, so convert them first:

./.venv/bin/python datasets/parquet_to_arrow.py

The converter scans under datasets/ and writes .arrow files next to matching .parquet files.

Observed local dataset splits:

Dataset	Split	Rows	Positive labels	Notes
R2Vul	train	14,678	7,193	Multilingual: C, Python, Java, JavaScript, C#.
R2Vul	val	1,818	887	Arrow file currently present.
R2Vul	test	1,838	906	Parquet currently present; convert for CL-AMP.
REVEAL	train	18,187	1,792	C-only, highly imbalanced.
REVEAL	val	2,273	224	Parquet currently present.
REVEAL	test	2,274	224	Parquet currently present.
VCLData strict	train	73,667	36,930	Synthetic/SARD-Juliet derived.
VCLData strict	val	9,208	4,616	Arrow file currently present.
VCLData strict	test	9,209	4,617	Use cautiously; see notes below.
PCVul single-format	train	18,160	9,080	Produced from paired data.
PCVul single-format	val	2,270	1,135	Produced from paired data.
PCVul single-format	test	2,272	1,136	Produced from paired data.
PrimeVul	val	23,948	593	Validation split only in current tree.

datasets/vcldata_data_leakage_analysis.md documents leakage and shortcut risks in VCLData/SARD-Juliet. Treat VCLData as diagnostic data rather than the primary benchmark.

CL-AMP Model

The implementation is in model.py.

Training inputs:

tokenized source code;
tokenized vulnerability/safety description;
binary label.

Code path:

text

code tokens
  -> sliding windows of length chunk_size
  -> UniXCoder hidden states
  -> per-window CLS vectors
  -> learned attention pooling
  -> residual simulator
  -> MLP classifier

Description-guided training path:

text

description tokens
  -> UniXCoder description embedding
  -> cross-attention over code window CLS vectors
  -> guided code representation

Current loss:

text

L_main  = CrossEntropy(MLP(V_hat), y)
L_align = InfoNCE(V_guided, D)
L_sim   = 1 - cosine_similarity(V_hat, stopgrad(V_guided))
L       = L_main + alpha * L_align + beta * L_sim

Inference uses source code only:

text

score = softmax(MLP(V_hat))[:, vulnerable]
prediction = score > threshold

Default model arguments are defined in clamp.Args:

Argument	Default
`pretrain_model_name`	`microsoft/unixcoder-base-nine`
`hidden_size`	`768`
`max_code_tokens`	`4096`
`chunk_size`	`512`
`window_stride`	`384`
`max_desc_tokens`	`2048`
`batch_size`	`16`
`lr`	`2e-5`
`num_epochs`	`50`
`patience`	`5`
`alpha`	`1.0`
`beta`	`0.3`
`tau`	`0.1`

CL-AMP CLI

Show top-level help:

./.venv/bin/python clamp.py --help

Train

Convert Parquet splits to Arrow first if the .arrow files are missing.

bash

./.venv/bin/python clamp.py train \
  --train_path datasets/train/r2vul_train.arrow \
  --val_path datasets/val/r2vul_val.arrow \
  --test_path datasets/test/r2vul_test.arrow \
  --save-checkpoint models/clamp_r2vul.pt \
  --pretrain-model-name microsoft/unixcoder-base-nine \
  --max_code_tokens 4096 \
  --chunk-size 512 \
  --window-stride 384 \
  --max-desc-tokens 2048 \
  --batch-size 16 \
  --lr 2e-5 \
  --num-epochs 50 \
  --patience 5 \
  --alpha 1.0 \
  --beta 0.3 \
  --tau 0.1

Training performs validation after each epoch, tunes the threshold on validation scores by F1, saves the best checkpoint, and then evaluates the best checkpoint on the configured test set.

Threshold Adaptation

Use this to load a checkpoint, tune the decision threshold on a validation set, and evaluate on a test set. This does not update model weights.

bash

./.venv/bin/python clamp.py adapt \
  --use-checkpoint models/clamp_r2vul.pt \
  --val_path datasets/val/r2vul_val.arrow \
  --test_path datasets/test/r2vul_test.arrow \
  --batch-size 16

Fixed-Threshold Evaluation

Use this when the threshold is already known:

bash

./.venv/bin/python clamp.py eval \
  --use-checkpoint models/clamp_r2vul.pt \
  --test_path datasets/test/r2vul_test.arrow \
  --threshold 0.2662 \
  --batch-size 16

Baselines

Baseline runners live under baselines/.

LineVul

./.venv/bin/python baselines/linevul_runner.py --help

The runner supports training and testing through --do_train and --do_test, using baselines/linevul_model.py.

ReGVD

./.venv/bin/python baselines/regvd_runner.py --help

The runner supports training and testing through --do_train and --do_test, using baselines/regvd_model.py.

CLeVeR

./.venv/bin/python baselines/clever_runner.py --help

The CLeVeR baseline has two stages:

contrastive pre-training with code and description pairs;
linear-probe training for vulnerability classification.

Example:

bash

./.venv/bin/python baselines/clever_runner.py \
  --do_train \
  --train_path datasets/train/r2vul_train.arrow \
  --val_path datasets/val/r2vul_val.arrow \
  --test_path datasets/test/r2vul_test.arrow \
  --output_dir models/clever_probe \
  --pretrain_code_model_name microsoft/codebert-base \
  --pretrain_text_model_name roberta-base \
  --code_length 512 \
  --desc_length 256 \
  --hidden_size 768 \
  --epochs 10 \
  --probe_epochs 10 \
  --train_batch_size 16 \
  --eval_batch_size 32

Gemma Prompt Baseline

./.venv/bin/python baselines/gemma_prompt_runner.py --help

This runner calls an OpenAI-compatible chat-completions endpoint. Configure the API endpoint and key through environment variables or .env.local. Do not commit secrets.

Logged Results

Representative logged results in this workspace:

Dataset	Model/log	Threshold	F1	Precision	Recall	Accuracy	AUC	PR-AUC	MCC
R2Vul	`logs/r2vul/clamp_r2vul.log`	0.2662	0.7835	0.7189	0.8609	0.7655	0.8422	0.8034	0.5425
REVEAL	`logs/reveal/clamp_reveal.log`	0.3916	0.5156	0.4825	0.5536	0.8975	0.8607	0.6001	0.4600

Logged PCVul paired result:

text

logs/paired/clamp_pcvul.log
P-C=0.1223, P-V=0.4164, P-B=0.4230, P-R=0.0383,
pair_order_acc=0.6240, pair_head_acc=0.5204

Older experiments and ablation logs are kept in archives/outputs/.

Metrics

util.compute_metrics reports:

F1
precision
recall
accuracy
ROC-AUC as auc
PR-AUC as pr_auc
MCC
confusion counts: tp, tn, fp, fn

util.tune_threshold selects the threshold that maximizes F1 on the validation scores.

LLM Description Augmentation

Use datasets/llm_augment.py to generate or fill description fields through an OpenAI-compatible API.

./.venv/bin/python datasets/llm_augment.py --help

The repository also contains baselines/gemma_prompt_runner.py for direct LLM classification. Both paths may read API credentials from environment variables or .env.local.

Reproducibility Checklist

For every experiment, record:

checkpoint path;
exact train/val/test dataset paths;
whether Parquet splits were converted to Arrow before running;
random seed;
model arguments: alpha, beta, tau, token limits, chunk size, window stride, batch size, learning rate;
tuned threshold;
full metrics output;
GPU type and PyTorch/CUDA versions;
log path.

There is no git repository detected in this workspace, so keep an external note of code/archive versions when preserving a result.

Citation

This repository is a research prototype for a thesis/project. Add formal citation details here when a paper or report is finalized.

dnx04/cl-amp

CL-AMP

Current Status

Repository Layout

Environment

Dataset Format

CL-AMP Model

CL-AMP CLI

Train

Threshold Adaptation

Fixed-Threshold Evaluation

Baselines

LineVul

ReGVD

CLeVeR

Gemma Prompt Baseline

Logged Results

Metrics

LLM Description Augmentation

Reproducibility Checklist

Citation

dnx04/
cl-amp