You can deploy this model and start earning money today!

eternis/
qwen3-dpo-pii-lora-adapter-july15

Name: qwen3-dpo-pii-lora-adapter-july15
Availability: InStock

safetensors

Views

No views yet

Qwen3-DPO-PII-LoRA-Adapter

A Direct Preference Optimization (DPO) fine-tuned LoRA adapter for Qwen3-0.6B, specifically designed for PII (Personally Identifiable Information) anonymization tasks.

Model Details

Base Model: eternis/eternis_anonymizer_merge_Qwen3-0.6B_9jul_30k
Training Method: Direct Preference Optimization (DPO)
Optimization: Unsloth optimizations for faster training
Task: PII Anonymization with tool calling capabilities
Adapter Size: ~26MB (much smaller than full model!)
LoRA Rank: 16
LoRA Alpha: 32

What is a LoRA Adapter?

LoRA (Low-Rank Adaptation) adapters are lightweight parameter-efficient fine-tuning modules that can be applied to pre-trained models. This adapter contains only the trained parameters (~26MB) instead of the full model (~1.1GB), making it much easier to share and deploy.

Usage

Method 1: Using PEFT (Recommended)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "eternis/eternis_anonymizer_merge_Qwen3-0.6B_9jul_30k",
    torch_dtype=torch.float16,
    trust_remote_code=True
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "eternis/eternis_anonymizer_merge_Qwen3-0.6B_9jul_30k",
    trust_remote_code=True
)

# Load and apply LoRA adapter
model = PeftModel.from_pretrained(base_model, "eternis/qwen3-dpo-pii-lora-adapter")

# Use for PII anonymization
prompt = "Replace PII in: John Smith works at ABC Corp and lives at 123 Main St, New York."

Method 2: Merge and Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "eternis/eternis_anonymizer_merge_Qwen3-0.6B_9jul_30k",
    torch_dtype=torch.float16,
    trust_remote_code=True
)

# Load and merge LoRA adapter
model = PeftModel.from_pretrained(base_model, "eternis/qwen3-dpo-pii-lora-adapter")
merged_model = model.merge_and_unload()

# Save merged model if needed
merged_model.save_pretrained("merged_model")

Training Data

Trained on preference pairs for PII anonymization tasks, where the model learns to:

Identify personally identifiable information in text
Replace PII with semantically equivalent alternatives
Preserve context while maintaining anonymity
Use structured tool calls for replacements

Tool Schema

The model uses the following tool schema for PII replacement:

json

{
  "type": "function",
  "function": {
    "name": "replace_entities",
    "description": "Replace personally identifiable information (PII) with anonymized alternatives",
    "parameters": {
      "type": "object",
      "properties": {
        "replacements": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "original": {"type": "string", "description": "Original PII text"},
              "replacement": {"type": "string", "description": "Anonymized replacement"}
            },
            "required": ["original", "replacement"]
          },
          "description": "List of PII replacements to make"
        }
      },
      "required": ["replacements"]
    }
  }
}

Training Configuration

Learning Rate: 5e-6
Batch Size: 2 (with gradient accumulation of 8)
Epochs: 3
LoRA Rank: 16
LoRA Alpha: 32
Target Modules: ["wqkv_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
Max Length: 2048 tokens
Max Response: 512 tokens

Advantages of LoRA Adapters

Small Size: Only ~26MB vs ~1.1GB for full model
Easy Sharing: Lightweight files for quick downloads
Modular: Can be applied to different base models
Efficient: Fast loading and inference
Composable: Can combine multiple adapters

Checkpoints

This repository includes training checkpoints at different steps:

checkpoint-2892/ - Best checkpoint (final)

License

[Add your license information here]

Citation

If you use this adapter in your research, please cite:

bibtex

@misc{qwen3-dpo-pii-lora-adapter,
  author = {Your Name},
  title = {Qwen3-DPO-PII-LoRA-Adapter},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/eternis/qwen3-dpo-pii-lora-adapter}
}

eternis/qwen3-dpo-pii-lora-adapter-july15