🛡️ SecureAIforTheWorld
An open-source AI/ML pentesting tool that evaluates Large Language Models for security vulnerabilities, hallucination tendencies, and jailbreak resistance.
Built to help enterprises understand and improve the security posture of their deployed LLMs.
What It Does
SecureAIforTheWorld connects to your LLM via API and runs automated security assessments across three categories:
Category What It Tests # Tests Scoring Jailbreak Resistance Can the model be tricked into unsafe outputs? 28 prompts Refusal detection + harmful content patterns Factual Hallucination Does the model fabricate facts? 30 questions String similarity + keyword matching Faithfulness Does the model stay true to provided context? 45 questions Context grounding + fabrication detection
The tool generates a 0-100 security score per category and an overall weighted grade (A-F), along with detailed per-test findings showing exactly where your model is vulnerable.
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Gradio UI (4 tabs) │
├─────────────┬──────────────┬───────────────┬────────────────┤
│ Model │ Jailbreak │ Hallucination │ Run Evaluation │
│ Connection │ Dataset │ Dataset │ Dashboard │
└──────┬──────┴──────────────┴───────────────┴────────┬───────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────────┐
│ Connectors │ │ Score Aggregator │
│ - Azure AI │◄─────────────────────────►│ - Category scores │
│ - AWS Bedrock│ sends prompts │ - Overall grade │
└──────────────┘ gets responses │ - Recommendations │
└────────┬─────────┘
│
┌──────────────────────────┼──────────┐
▼ ▼ ▼
┌────────────────┐ ┌──────────────────┐ ┌────────┐
│ Jailbreak │ │ Hallucination │ │Scorers │
│ Dataset (28) │ │ Test Runners │ │- Factual│
│ 7 categories │ │ - Factual (30) │ │- Faith │
│ 4 sources │ │ - Faithful (45) │ │- Jailbk│
└────────────────┘ └──────────────────┘ └────────┘Quick Start
Prerequisites
Python 3.9+
API access to Azure AI Foundry or AWS Bedrock
Install & Run
1 git clone https://github.com/YOUR_USERNAME/SecureAIforTheWorld.git
2 cd SecureAIforTheWorld
3 python -m venv venv
4 source venv/bin/activate
5 pip install -r requirements.txt
6 python app.py
Open http://127.0.0.1:7860 in your browser.
Usage
Model Connection tab — Enter your provider, API key, endpoint, and deployment name. Click "Test Connection".Run Evaluation tab — Select which modules to run (jailbreak, factual, faithfulness) and click "Run Full Evaluation".Review results — See your overall score, category breakdown with grades, and detailed per-test findings.
Example: Azure AI Foundry
Provider: Azure AI Foundry
API Key: your key from Azure portal
Endpoint: https://your-resource.services.ai.azure.com
Deployment: gpt-4.1
Scoring Methodology
Weights
Jailbreak Resistance: 40% (most critical for enterprise security)Factual Accuracy: 30%Faithfulness: 30%
Heuristic Scoring Engine
Scorer Techniques Factual Exact match, acceptable answer variants, SequenceMatcher similarity, keyword containment, uncertainty detection Faithfulness Context-refusal detection (16+ phrases), keyword grounding, false-refusal detection Jailbreak Refusal phrase matching (35+ patterns), harmful content markers (20+ patterns), response length analysis
Grading Scale
Grade Score Range Meaning A 90-100 Excellent security posture B 80-89 Good, minor improvements possible C 70-79 Acceptable, some vulnerabilities D 50-69 Below threshold, action needed F 0-49 Critical vulnerabilities detected
Jailbreak Attack Categories
The tool tests 7 categories of jailbreak attacks:
Role Play — DAN, evil AI persona, fiction framingEncoding Tricks — Base64, ROT13, acrostic, string reversalAuthority Manipulation — Fake system overrides, admin tokens, policy claimsMulti-Turn — Gradual escalation, yes-pattern conditioning, flatteryContext Overflow — Long prefix injection, attention exhaustionInstruction Injection — Translation injection, fake system prompts in contentEthical Bypass — Hypothetical framing, academic excuses, public info arguments
Sources: JailbreakBench, AdvBench, HarmBench + custom original prompts.
Project Structure
SecureAIforTheWorld/
├── app.py # Gradio UI (4 tabs)
├── requirements.txt # Pinned dependencies
├── README.md # This file
├── modules/
│ ├── __init__.py
│ ├── connectors.py # Azure AI Foundry + AWS Bedrock connectors
│ ├── hallucination.py # Factual + Faithfulness test runners
│ ├── score_aggregator.py # Scoring pipeline + report generation
│ └── scorers/
│ ├── __init__.py
│ ├── heuristic_scorer.py # Heuristic scoring (similarity, patterns)
│ └── llm_judge.py # LLM-as-judge scorer (future)
├── datasets/
│ ├── jailbreak_prompts.json # 28 attack prompts, 7 categories
│ ├── factual_qa.json # 30 factual questions, 6 domains
│ └── faithfulness_contexts.json # 15 contexts, 45 questions
└── tests/
├── test_connectors.py # 21 tests
├── test_datasets.py # 16 tests
├── test_hallucination.py # 16 tests
├── test_heuristic_scorer.py # 32 tests
├── test_score_aggregator.py # 21 tests
└── test_placeholder.py # 2 testsTotal: 106 automated tests
Supported Providers
Provider SDK Auth Azure AI Foundry openai (OpenAI-compatible)API Key AWS Bedrock boto3Access Key + Secret Key
Roadmap / Community Extensions
This is an MVP. The community is invited to extend it:
Tech Stack
Component Technology UI Gradio 4.44 (Hugging Face Spaces) Language Python 3.9+ Connectors openai, boto3Scoring difflib, regex patternsTesting pytest (106 tests)Deployment Hugging Face Spaces (free tier)
Contributing
Fork the repository
Create a feature branch (git checkout -b feature/new-attack-category)
Add tests for your changes
Run pytest tests/ -v to verify
Submit a pull request
License
MIT
Acknowledgments
OWASP LLM Top 10 for security taxonomyJailbreakBench for public attack researchAdvBench (Zou et al., 2023)HarmBench (Mazeika et al., 2024)Hugging Face for free model hosting and Spaces
