User Submissions by Detection Method - Spam (FP)

Hunt Hypothesis

This hypothesis targets potential phishing campaigns (T1566) where users are reporting messages that were initially classified as spam, indicating a need to validate if these are genuine false positives or missed malicious emails. Proactively hunting for these submissions in Azure Sentinel allows the SOC to refine spam filter accuracy and identify emerging phishing vectors that bypass initial automated defenses.

KQL Query

let Submissions = CloudAppEvents
    | where Timestamp > ago(30d)
    | where ActionType == "UserSubmission"
    | extend RawData = parse_json(RawEventData)
    | where tostring(RawData.SubmissionContentType) == "Mail" and tostring(RawData.SubmissionType) == "3"
    | extend NetworkMessageId = tostring(RawData.ObjectId), RecipientObjectId = AccountObjectId
    | distinct NetworkMessageId, RecipientObjectId;
EmailEvents
| where Timestamp > ago(30d)
| join kind=inner Submissions on NetworkMessageId, RecipientObjectId
// De-duplicate to the latest EmailEvents record per message and recipient so the verdict is read once
| summarize arg_max(Timestamp, *) by NetworkMessageId, RecipientObjectId
| where EmailDirection == "Inbound" and DetectionMethods has 'Spam'
| mv-expand Spam = parse_json(DetectionMethods).Spam to typeof(string)
| where isnotempty(Spam)
| summarize count() by Spam
| render piechart

Analytic Rule Definition

id: 91a1a3f0-1386-4c70-9c66-215f59b2af28
name: User Submissions by Detection Method - Spam (FP)
description: |
  This query visualises user false positive submissions by the original spam filter verdict on the reported message.
description-detailed: |
  This query visualises user false positive submissions by the original spam filter verdict on the reported message.
  Query is also included as part of the Defender for Office 365 solution in Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-2-build-custom-email-security-reports-and-dashboards-with-workbooks-in-micr/4411303
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
  dataTypes:
  - CloudAppEvents
  - EmailEvents
tactics:
  - InitialAccess
relevantTechniques:
  - T1566
query: |
  let Submissions = CloudAppEvents
      | where Timestamp > ago(30d)
      | where ActionType == "UserSubmission"
      | extend RawData = parse_json(RawEventData)
      | where tostring(RawData.SubmissionContentType) == "Mail" and tostring(RawData.SubmissionType) == "3"
      | extend NetworkMessageId = tostring(RawData.ObjectId), RecipientObjectId = AccountObjectId
      | distinct NetworkMessageId, RecipientObjectId;
  EmailEvents
  | where Timestamp > ago(30d)
  | join kind=inner Submissions on NetworkMessageId, RecipientObjectId
  // De-duplicate to the latest EmailEvents record per message and recipient so the verdict is read once
  | summarize arg_max(Timestamp, *) by NetworkMessageId, RecipientObjectId
  | where EmailDirection == "Inbound" and DetectionMethods has 'Spam'
  | mv-expand Spam = parse_json(DetectionMethods).Spam to typeof(string)
  | where isnotempty(Spam)
  | summarize count() by Spam
  | render piechart
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: InitialAccess
• Technique: T1566 — T1566

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Submissions/User Submissions by Detection Method - Spam FP.yaml)

False Positive Guidance

• Marketing Automation & Newsletter Campaigns: Legitimate bulk emails from tools like Mailchimp, HubSpot, or SendGrid often contain high link density or trigger heuristic spam scores due to template structures. Users may flag these as “Spam” because they are unsolicited or promotional in nature, even if the sender is whitelisted.
  • Filter/Exclusion: Exclude messages where the From domain matches known marketing platforms (e.g., @sendgrid.net, @hubspot.net) AND the Subject contains campaign keywords (e.g., “Newsletter”, “Weekly Digest”, “Promo”).
• System Notifications & Alerting Services: Automated alerts from monitoring and CI/CD tools such as Datadog, PagerDuty, Jira, or GitHub Actions are frequently flagged by users who perceive them as noise or “junk” mail, especially if the sender address is a generic alias (e.g., no-reply@...).
  • Filter/Exclusion: Exclude messages where the From address contains no-reply, alerts, notifications, or system AND the MIME-Type is text/plain (indicating simple system output rather than complex HTML marketing).
• Scheduled Reporting & Data Exports: Recurring reports generated by Power BI, Tableau, SAP, or Oracle database systems are often sent to distribution lists. Users may mark these as spam if the report is large, contains many attachments, or if the sender is a shared mailbox that lacks a personal name.
  • Filter/Exclusion: Exclude messages where the Subject starts with “Report”, “Daily Summary”, or “Weekly KPI” AND the AttachmentCount is greater than 0, or where the From domain is a known internal reporting server (e.g., `