This hypothesis targets adversaries leveraging sophisticated, AI-generated phishing content that evades traditional signature-based detection, as identified by Microsoft Defender for Office 365’s LLM analysis. Proactively hunting for these events allows the SOC to identify high-fidelity social engineering attempts early, reducing the risk of credential theft or malware delivery before user interaction occurs.
// OrgLevelPolicy exclusions drop phishing-simulation and SecOps-mailbox deliveries; remove them to include those.
EmailEvents
| where Timestamp > ago(30d)
| where OrgLevelPolicy != "Phishing simulation" and OrgLevelPolicy != "SecOps Mailbox"
| extend Key = strcat(NetworkMessageId, '-', RecipientEmailAddress)
| summarize arg_max(Timestamp, *) by Key
| where ThreatTypes has "Phish" and tostring(DetectionMethods) has "LLM content analysis"
| project Timestamp, SenderFromAddress, RecipientEmailAddress, Subject, ThreatTypes, DetectionMethods, NetworkMessageId
| top 100 by Timestamp desc
id: cf9cbcb0-5b9d-4054-8342-c47168ea50c6
name: Phishing Detected by LLM Content Analysis
description: |
This query lists inbound phishing messages that Microsoft Defender for Office 365 detected using LLM (Language AI) content analysis, using the EmailEvents table.
description-detailed: |
Microsoft Defender for Office 365 uses Language AI (LLM) content analysis to detect phishing and business email compromise from the content of a message. This query lists the most recent inbound emails classified as Phish where LLM content analysis contributed to the detection, so analysts can see what this newer detection method is catching and triage the messages.
This query is part of the Microsoft Defender for Office 365 Detections and Insights workbook in Microsoft Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-3-build-custom-email-security-reports-with-power-bi-and-workbooks-in-micros/4490127
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
dataTypes:
- EmailEvents
tactics:
- InitialAccess
relevantTechniques:
- T1566
query: |
// OrgLevelPolicy exclusions drop phishing-simulation and SecOps-mailbox deliveries; remove them to include those.
EmailEvents
| where Timestamp > ago(30d)
| where OrgLevelPolicy != "Phishing simulation" and OrgLevelPolicy != "SecOps Mailbox"
| extend Key = strcat(NetworkMessageId, '-', RecipientEmailAddress)
| summarize arg_max(Timestamp, *) by Key
| where ThreatTypes has "Phish" and tostring(DetectionMethods) has "LLM content analysis"
| project Timestamp, SenderFromAddress, RecipientEmailAddress, Subject, ThreatTypes, DetectionMethods, NetworkMessageId
| top 100 by Timestamp desc
version: 1.0.0
| Sentinel Table | Notes |
|---|---|
EmailEvents | Ensure this data connector is enabled |
Automated Ticketing System Notifications: Legitimate ITSM platforms (e.g., ServiceNow, Jira, or Zendesk) often send highly narrative, context-rich emails for complex incident updates or change requests. These emails may contain informal language, long paragraphs, or specific technical jargon that the LLM misinterprets as a “conversational” phishing lure rather than a structured notification.
Sender address matches known ITSM service accounts (e.g., no-reply@servicenow.com, jira-bot@company.com) or where the Subject line contains standard prefixes like [INC-, [CHG-, or [ALERT-.Internal Knowledge Base or Wiki Change Alerts: Tools like Confluence, SharePoint, or Notion send automated “page updated” or “comment added” notifications. These emails frequently include snippets of the edited content, which can be unstructured, contain markdown artifacts, or include user-generated text that lacks the rigid formatting of standard corporate communications, potentially triggering LLM heuristics for social engineering.
Sender is a known internal bot account (e.g., confluence-bot@company.com) or where the MessageId header contains specific identifiers used by the internal wiki platform.Marketing or Newsletter Digests from SaaS Vendors: Many enterprise SaaS providers (e.g., Slack, Zoom, GitHub) send weekly or monthly activity digests. These emails often use engaging, marketing-style language, emojis, or informal greetings (“Hey team! Here’s what you missed…”) to improve open rates. The LLM may classify this friendly, non-transactional tone as a sign of a phishing attempt aimed at building rapport.