Azure OpenAI Content Filtering and Abuse Monitoring with Microsoft Sentinel
Nogganna work here...
This topic has come up a lot recently - and it seems like more and more customers want to know if Microsoft Sentinel can be used as a Content Filtering or Abuse Monitoring mechanism for Azure OpenAI.
Microsoft Sentinel, of course, is a powerful tool for capturing and analyzing logs. And, while its intent is to analyze for security value, it can be used to analyze and report anything as long as it exists in the log data. Personally, I recommend against using it for anything but security. There are reasons. Good ones. But something to cover in another topic.
Unfortunately, user prompts are not captured in the standard logs. They are captured as part of the Content Filtering feature in Azure AI Studio, but the data is not exposed outside of the feature and is only retained for 30 days. And since it’s not exposed outside of the feature, it will not be available to be consumed by Microsoft Sentinel. Sorry.
What is a prompt? See: Must Learn AI Security Part 1: Prompt Injection Attacks Against AI for a definition
There are reasons why this data is not made available to a SIEM. The content that is collected through Content Filtering and Abuse Monitoring can be super-sensitive data - the kind that HR freaks out about, for example.
Like much with AI these days, this topic is constantly changing, so this is the answer for now. If it changes, I’ll update the messaging here.
In the interim, learn more about Azure OpenAI’s Content Filtering and Abuse Monitor capabilities in the following...
Content Filtering
This system works by running both the prompt and completion through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions. In addition to the content filtering system, the Azure OpenAI Service performs monitoring to detect content and/or behaviors that suggest use of the service in a manner that may violate applicable product terms.
Content filtering: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter
Abuse Monitoring
Abuse monitoring detects and mitigates instances of recurring content and/or behaviors that suggest use of the service in a manner that may violate the Code of Conduct or other applicable product terms. The service employs several components to abuse monitoring, including content classification, abuse pattern capture, human review and decision, and notification and action.
Abuse monitoring: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/abuse-monitoring
Want more? I invite Richard “Disney” Diver to an episode of “After the Blog” and take this topic a bit further.
[Want to discuss this further? Hit me up on Twitter or LinkedIn]
[Subscribe to the RSS feed for this blog]
[Subscribe to the Weekly Microsoft Sentinel Newsletter]
[Subscribe to the Weekly Microsoft Defender Newsletter]
[Subscribe to the Weekly Azure OpenAI Newsletter]
[Learn KQL with the Must Learn KQL series and book]
[Learn AI Security with the Must Learn AI Security series and book]