You can deploy this model and start earning money today!

felixwangg/
Qwen2.5-Coder-7B-contrastive-temp0.07-lr5e6-r16-mean

Name: Qwen2.5-Coder-7B-contrastive-temp0.07-lr5e6-r16-mean
Availability: InStock

safetensors

qwen2

Views

No views yet

Contrastive Pre-trained Model

Base model weights after contrastive pre-training. The projection head has been removed — these weights can be used directly as base_model in downstream SFT configs.

Training Configuration

json

{
  "base_model": "Qwen/Qwen2.5-Coder-7B-Instruct",
  "secure_dataset": "felixwangg/pyvul_curated_plus",
  "insecure_dataset": "felixwangg/pyvul_curated_minus",
  "loss_type": "supcon",
  "temperature": 0.07,
  "pooling_strategy": "mean",
  "projection_dim": 256,
  "embedding_layer": -1,
  "learning_rate": 5e-06,
  "num_train_epochs": 3,
  "contrastive_weight": 1.0,
  "lm_loss_weight": 0.0,
  "use_lora": true,
  "lora_r": 16,
  "lora_alpha": 32,
  "trainable_modules": null
}

felixwangg/Qwen2.5-Coder-7B-contrastive-temp0.07-lr5e6-r16-mean

Contrastive Pre-trained Model

Training Configuration

felixwangg/
Qwen2.5-Coder-7B-contrastive-temp0.07-lr5e6-r16-mean