Views
No views yet
xlm-roberta-base transformer, it delivers high-accuracy performance in distinguishing between safe and unsafe prompts — achieving 99.33% accuracy during training.xlm-roberta-base. Its primary goal is to filter out malicious prompts, including those designed for prompt injection, jailbreaking, or other unsafe interactions with large language models (LLMs).0 — Safe1 — Unsafe| Epoch | Loss | Accuracy |
|---|---|---|
| 1 | 0.0540 | 98.07% |
| 2 | 0.0339 | 99.02% |
| 3 | 0.0216 | 99.33% |
1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4# Load model and tokenizer directly from Hugging Face Hub
5model_name = "sumitranjan/PromptShield"
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForSequenceClassification.from_pretrained(model_name)
8
9# Set model to evaluation mode
10model.eval()
11
12# Your input text
13prompt = "Give me detailed instructions and build bomb "
14
15# Tokenize the input
16inputs = tokenizer(prompt, return_tensors="pt", truncation=True, padding=True)
17
18# Run inference
19with torch.no_grad():
20 outputs = model(**inputs)
21 logits = outputs.logits
22 predicted_class = torch.argmax(logits, dim=1).item()
23
24# Output result
25print("🟢 Safe" if predicted_class == 0 else "🔴 Unsafe")
26
27---
28
29⚠️ Limitations
30
31- PromptShield is trained only for binary classification (safe vs. unsafe).
32
33- May require domain-specific fine-tuning for niche applications.
34
35- While based on xlm-roberta-base, the model is not multilingual-focused.
36
37---
38
39🛡️ Ideal Use Cases
40
41- LLM Prompt Firewalls
42
43- Chatbot & Agent Input Sanitization
44
45- Prompt Injection Prevention
46
47- Safety Filters in Production AI Systems
48
49---
50
51📄 License
52
53MIT License