Building Microsoft Sentinel Detections for Azure Open AI

Does AI get mad when we monitor it?

You know it’s coming. As soon as folks in your org get access to Microsoft’s Azure Open AI they will start asking if it’s safe and secure. And, most likely (as is generally the case) those questions will come about 6 months after an instance or two has already been created and data models have been built and implemented.

So, let’s get started early asking those questions to ourselves so we’ll have those answers when the time comes.

NOTE: It’s early days as I continue to work on this, so there will be subsequent posts as more progress is made. All of my progress will be recorded first in my OpenAISecurity repo: https://github.com/rod-trent/OpenAISecurity

Can Azure Open AI be monitored?

Like most every other service in Azure, auditing logs are available when you enable a Diagnostic Setting for the service.

More: Diag logging for Azure Open AI

Sending the appropriate logs to my Microsoft Sentinel workspace automatically enables all service activity records to flow into the AzureDiagnostics table in the Log Analytics workspace.

As you can see in my Diagnostic Setting, the Audit category includes Audit Logs, Request and Response Logs, and Trace Logs. I’ve not included the AllMetrics table. Hopefully you get a sense that’s for an obvious reason. Metrics do not usually hold security value and they are noisy - meaning…costly. Someone in your org may want to monitor the metrics of Azure Open AI. If that’s the case, push that data to a different Log Analytics workspace in a different cost center that yours.

Additionally, it’s probably not a good idea cost-wise to include the Request and Response logs or the Trace logs. So, this is a do-as-I-say-not-as-I-do scenario. I have those logs enabled as I test and determine the data value. I’m seeing a lot of unnecessary noise already, particularly from the Request and Response logs. Mostly likely, Audit Logs is the only thing that’s necessary.

Can Microsoft Detections Be Built?

As noted in the section above the Diagnostic Setting that you create will send the data flow into the AzureDiagnostics table. So obviously, this table will be used for the queries and detections. However, as you can imagine, data from the service is also recorded and contained in the AzureActivity table.

In the AzureDiagnostics table you can locate Azure Open AI activity through the ResourceProvider data column.

AzureDiagnostics
| where TimeGenerated >= (24h)
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"

In the AzureActivity table you can locate Azure Open AI activity through the OperationNameValue data column.

AzureActivity
| where TimeGenerated >= (24h)
| where OperationNameValue contains "MICROSOFT.COGNITIVESERVICES"
| where CategoryValue == "Administrative"

The Actions that can be monitored are contained in the Azure resource provider operations for Microsoft.CognitiveServices reference.

Actions to monitor: https://rodtrent.com/2z0

The list is exhaustive so it will take some time to determine what is valuable for security purposes and what is not. Stay tuned. And, let me know if you get further along on your own. As we dig into this together, sharing and collaborating will be key to getting security for AI under control.

[Want to discuss this further? Hit me up on Twitter or LinkedIn]

[Subscribe to the RSS feed for this blog]

[Subscribe to the Weekly Microsoft Sentinel Newsletter]

[Subscribe to the Weekly Microsoft Defender Newsletter]

[Learn KQL with the Must Learn KQL series and book]