User Email Submissions (FN-FP) by Grading verdict

Hunt Hypothesis

Adversaries may submit malicious emails to evade detection, leveraging false negatives or positives to bypass security controls. SOC teams should proactively hunt for this behavior in Azure Sentinel to identify potential evasion tactics and refine detection rules.

KQL Query

CloudAppEvents 
| where ActionType contains "UserSubmissionTriage" 
| extend UserKey = (parse_json(RawEventData)).UserKey, SubmissionState = (parse_json(RawEventData)).SubmissionState,  SubmissionId=(parse_json(RawEventData)).SubmissionId, TriageVerdict=(parse_json(RawEventData)).GradingResult.TriageVerdict 
| summarize count() by tostring(TriageVerdict)
| render piechart

Analytic Rule Definition

id: 3b9e6da0-3504-463f-80d3-31fe3f0261ff
name: User Email Submissions (FN-FP) by Grading verdict
description: |
  This query visualises the total ammount of user false negative or false positive submissions by the verdict of the submission grading.
description-detailed: |
  This query visualises the total ammount of user false negative or false positive submissions by the verdict of the submission grading.
  Query is also included as part of the Defender for Office 365 solution in Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-2-build-custom-email-security-reports-and-dashboards-with-workbooks-in-micr/4411303
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
  dataTypes:
  - CloudAppEvents
tactics:
  - InitialAccess
relevantTechniques:
  - T1566
query: |
  CloudAppEvents 
  | where ActionType contains "UserSubmissionTriage" 
  | extend UserKey = (parse_json(RawEventData)).UserKey, SubmissionState = (parse_json(RawEventData)).SubmissionState,  SubmissionId=(parse_json(RawEventData)).SubmissionId, TriageVerdict=(parse_json(RawEventData)).GradingResult.TriageVerdict 
  | summarize count() by tostring(TriageVerdict)
  | render piechart
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: InitialAccess
• Technique: T1566 — T1566

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Submissions/User Submissions by Grading Verdict - FN-FP.yaml)

False Positive Guidance

• Scenario: Scheduled Email Archiving Job
  Description: A scheduled job runs nightly to archive user emails to a compliance storage system. This job may submit emails to the grading system as part of its process.
  Filter/Exclusion: Exclude submissions where the source is a known archiving tool (e.g., Microsoft Exchange Archiving Service or third-party tools like Symantec Enterprise Vault).

• Scenario: User-Initiated Email Export to CSV
  Description: A user manually exports their email inbox to a CSV file using a tool like Microsoft Outlook or a webmail interface. This action may trigger submission events that are flagged by the grading system.
  Filter/Exclusion: Exclude submissions where the user agent or source IP matches known email client tools (e.g., Outlook, Thunderbird) or where the file extension is .csv.

• Scenario: System-Wide Email Backup Process
  Description: An automated backup process (e.g., Veeam, Commvault, or native email server backups) submits emails to the grading system as part of its backup workflow.
  Filter/Exclusion: Exclude submissions where the source is a backup tool (e.g., Veeam Backup & Replication, Commvault) or where the submission is associated with a backup schedule.

• Scenario: Admin Task for Email Compliance Review
  Description: An admin manually reviews emails for compliance purposes using a tool like Microsoft 365 Compliance Center or a third-party eDiscovery tool. This may result in submissions being graded.
  Filter/Exclusion: Exclude submissions where the user is a known admin or where the submission is associated with a compliance task (e.g., compliance-review or eDiscovery in the task description).

• Scenario: Email Submission via API for Integration Testing
  Description: A development team uses an API (e.g., Microsoft Graph API,