Using Microsoft Sentinel to Monitor, Detect and Alert Bad AI Content

The Ian Malcolm Effect

This is one of those “just because you can, should you?” scenarios, but there’s been some discussion recently around how to effectively use Microsoft Sentinel as an AI data filtering and alerting mechanism.

Obviously, for those utilizing Azure OpenAI, there’s the Content Filtering mechanism built in, which is the recommended way to handle content filtering.

But there are some Microsoft Sentinel customers interested in utilizing the power of Microsoft Sentinel to ingest, retain, and analyze the data. I don’t recommend this for one obvious reason, and that is… Do we really want our security teams focused on non-security activities that might make them miss an actual security threat? Probably not. The first time the security team is caught handling a bad word in a user’s AI prompt while a threat actor is successfully taking down the entire network and slapping a ransomware tag on all corporate data, things will definitely change.

Forbidden content is an organization policy/HR issue. But I get it. Microsoft Sentinel is super powerful and can definitely do the job. And, if the alerts are routed to HR instead of the security team, then you might have something.

Here’s how this might work:

1. Ingest the AI data (activity logs) into Microsoft Sentinel into a custom table. Activity logs could come from things like the AI service, or the AI app.

2. Create a Watchlist that contains organizational determined bad content. To get started one could import the List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words (used by orgs like Slack and Discord) into a Watchlist and modify the list on an ongoing basis as policies change.

3. Write an Analytics Rule to alert when bad content is detected in the custom table.

A simple KQL query example:

let BadContent = (_GetWatchlist('BadStuff') | project Bad_Words);
YourCustomAIDataTable_CL
| where prompt_data has_any (BadContent)

4. Give Responder access to Microsoft Sentinel to those assigned to deal with AI bad content and use an Automation Rule to automatically route it to them.

Additionally, a Workbook or a PowerBI report could be created to better identify infractions on a larger scale.

Again, I’m in no way recommending using a SIEM for AI content filtering and review, but this can be done. If you’ve been following along with the Must Learn AI Security series, you know that bad content can lead to a compromised AI. And there’s more and more talk about combining Responsible AI and Security of AI into a single “Safe AI” branch, so who knows?

If you decide to go down this path - even just for testing - I’d love to hear from you.

---

[Want to discuss this further? Hit me up on Twitter or LinkedIn]

[Subscribe to the RSS feed for this blog]

[Subscribe to the Weekly Microsoft Sentinel Newsletter]

[Subscribe to the Weekly Microsoft Defender Newsletter]

[Subscribe to the Weekly Azure OpenAI Newsletter]

[Learn KQL with the Must Learn KQL series and book]

[Learn AI Security with the Must Learn AI Security series and book]