Phishing Detection via Reasoning LLM

Why Phishing Matters?

• Phishing attacks are becoming more widespread due to the rapid growth of the internet.
• These attacks cause billions of dollars in losses every year.
• Traditional research has relied on:
  • Statistical methods
  • Transformer models
• While these methods achieve strong predictive accuracy, they lack clear justifications for their classifications.

Enter Large Language Models (LLMs)

• LLMs show strong potential for textual analysis.
• Especially promising are reasoning-based LLMs:
  • They can break down complex problems into step-by-step reasoning.
• This study explores fine-tuning LLMs for phishing and scam detection using the Qwen3-8B model.

Research Focus

1. Training Methods

• Supervised Fine-Tuning (SFT): mimics expert-labelled data.
• Guided Reinforcement Learning (GRPO): explores and adapts through self-improvement.

1. Model Starting Point

• Fine-tuning a raw base model.
• Fine-tuning an instruction-aware assistant (already aligned to follow directions).

1. Verification Layer

• Adding a verifier to refine or correct the model’s first response.

Evaluation & Dataset

• Models were tested against:
  • ML methods (like logistic regression)
  • BERT and ModernBERT
  • Other proprietary LLMs (like OpenAI and Gemini) and open-source LLMs (DeepSeek R1 and Qwen3)
• A new dataset was created from a public scam-reporting forum to ensure recency and relevance.

Key Findings

1. SFT vs GRPO

• SFT: Higher recall (catches more phishing attempts).
• GRPO: Higher precision (reduces false positives).
• Trade-off: sensitivity vs reliability.

1. Starting Point Matters

• Beginning with an instruction-tuned model is critical for success.

1. Verifier Effects

• A verifier doesn’t boost accuracy overall.
• Instead, it acts as a “specialisation amplifier”, reinforcing each model’s natural strengths and weaknesses.

Takeaways

• Fine-tuned open-source LLMs still trail behind simple ML models in raw performance.
• However, they excel in providing transparent, context-based justifications for their classifications.
• Proprietary LLMs outperform all tested models, showing that with the right methods, LLMs can:
  • Accurately detect fraudulent texts
  • Explain their reasoning
• This opens a promising direction for future phishing detection research.

Results

(Read the paper for the full results and analysis.)

Usage