Admin Submissions by Grading verdict (FN-FP)

Hunt Hypothesis

Adversaries may exploit false negative or false positive grading verdicts to bypass detection and maintain persistence within the environment. SOC teams should proactively hunt for this behavior in Azure Sentinel to identify potential evasion tactics and refine detection rules.

KQL Query

CloudAppEvents
| where ActionType contains "AdminSubmissionTriage"
| extend UserKey = (parse_json(RawEventData)).UserKey, SubmissionState = (parse_json(RawEventData)).SubmissionState,  SubmissionId=(parse_json(RawEventData)).SubmissionId, TriageVerdict=(parse_json(RawEventData)).GradingResult.TriageVerdict
| summarize count() by tostring(TriageVerdict)
| render piechart

Analytic Rule Definition

id: f7a36ec9-f708-420c-b248-7dfe6cc39d0a
name: Admin Submissions by Grading verdict (FN-FP)
description: |
  This query visualises the total amount of admin false negative or false positive submissions by the verdict of the submission grading.
description-detailed: |
  This query visualises the total amount of admin false negative or false positive submissions by the verdict of the submission grading.
  Query is also included as part of the Defender for Office 365 solution in Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-2-build-custom-email-security-reports-and-dashboards-with-workbooks-in-micr/4411303
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
  dataTypes:
  - CloudAppEvents
tactics:
  - InitialAccess
relevantTechniques:
  - T1566
query: |
  CloudAppEvents
  | where ActionType contains "AdminSubmissionTriage"
  | extend UserKey = (parse_json(RawEventData)).UserKey, SubmissionState = (parse_json(RawEventData)).SubmissionState,  SubmissionId=(parse_json(RawEventData)).SubmissionId, TriageVerdict=(parse_json(RawEventData)).GradingResult.TriageVerdict
  | summarize count() by tostring(TriageVerdict)
  | render piechart
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: InitialAccess
• Technique: T1566 — T1566

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Submissions/Admin Submissions by Grading Verdict - FN-FP.yaml)

False Positive Guidance

• Scenario: Scheduled system backup job submits a false positive grading
  Filter/Exclusion: process.name != "backup-service" or process.name != "system-backup"

• Scenario: Admin manually reviews and submits a false positive via the SIEM console
  Filter/Exclusion: process.name != "siem-console" or user.name != "admin-username"

• Scenario: Automated log parsing tool flags benign entries as false positives during ingestion
  Filter/Exclusion: process.name != "log-parsing-tool" or process.name != "log-processor"

• Scenario: User submits a false positive via the endpoint detection and response (EDR) tool’s feedback feature
  Filter/Exclusion: process.name != "edr-feedback-tool" or process.name != "endpoint-feedback"

• Scenario: Admin runs a compliance scan that generates false positives due to outdated rule sets
  Filter/Exclusion: process.name != "compliance-scan" or process.name != "audit-tool"