You can deploy this model and start earning money today!

durinn/
data-eval

Name: data-eval
Rating: 2.5 (8 reviews)

safetensors

deberta-v2

Views

No views yet

🧠 DeBERTa-v3-Base Code Quality Classifier

A fine-tuned DeBERTa-v3-base model trained to classify clean vs. buggy code using the CodeXGlue Defect Detection dataset.
This model is designed specifically for dataset filtering to improve downstream code language model training (e.g., Qwen2.5-Coder).

📌 Model Summary

This classifier predicts whether a given code snippet is non-defective (label 0) or buggy (label 1).
The output probabilities are used to rank samples by quality and select the highest-quality subset.

This model is part of a research pipeline analyzing how data quality affects token-level performance in generative code models.

Expected Result

5% improvement in perplexity based on early benchmarking of tuned DeBERTa

🧱 Model Description

Architecture

Base model: microsoft/deberta-v3-base
Task: Binary sequence classification
Labels:
- 0 = clean code
- 1 = buggy / defective code
Max sequence length: 512 tokens

Purpose

This model is intended for:

Dataset quality filtering
Improving generative model training stability
Research on LLM token quality and perplexity
Understanding effects of removing noisy samples

This model is not intended for real-world bug detection or vulnerability scanning.

📚 Dataset

Training Dataset

CodeXGlue Code Defect Detection
(https://huggingface.co/datasets/code_x_glue_cc_defect_detection)

"func": raw function-level source code
"target": binary label (0 = clean, 1 = buggy)
~21,000 training examples

Preprocessing

Tokenized with DeBERTa-v3-base tokenizer
Truncated to 512 tokens
Padded dynamically using DataCollatorWithPadding

🧪 Training Procedure

Hyperparameters

Hyperparameter	Value
Epochs	1
Learning rate	2e-5
Batch size	8
FP16	Yes
Max length	512
Optimizer	AdamW
Loss	Cross-entropy
remove_unused_columns	False

Training Code Snippet

python

model = AutoModelForSequenceClassification.from_pretrained(
    "microsoft/deberta-v3-base", num_labels=2
)

TrainingArguments(
    output_dir="filter_model",
    learning_rate=2e-5,
    num_train_epochs=1,
    fp16=True,
    per_device_train_batch_size=8,
    logging_steps=50,
    remove_unused_columns=False,
)

durinn/data-eval