Building Your own Conversational Copilot with Python, Flask, and Azure Open AI SDK
Chatting to oneself is a sign of something
If you’ve been reading here for very long, you know that I’ve been on a trek to discover the best practices for monitoring security for AI - particularly for Azure Open AI and Azure Cognitive services.
You can find the growing list of things to monitor in the Microsoft Sentinel area of the Open AI Security GitHub Repo here: https://github.com/rod-trent/OpenAISecurity/tree/main/Security/Sentinel
I started the trek working with the public ChatGPT and Open AI, but the more I learned there, the more I knew I needed to be working with Azure’s own responsible and secure guardrails for AI. I’ll dig deep into the reasons Enterprises should avoid public Open AI (and avoid allowing their users to use it, too) and stick with Azure Open AI in a future post.
The Code
To build data to determine what AI data to be monitored, I’ve had to develop some apps. As I noted when I released my command-line Chatbot, I am no programmer. However, I’ve had to buckle down a bit and figure out how best to show AI activity. So, my trek has also taken me down the developer road, in particularly using Python, Flask, and other tools.
My latest attempt has resulted in a web based Chatbot, much like you’ll see from many of the Microsoft Copilot iterations. All the code is located here: https://github.com/rod-trent/OpenAISecurity/tree/main/Code/Web%20Chat%20Bot
Here’s the files that are required from the repo:
WebChatBot.py - this is the primary app. Place this into the app’s primary working directory.
index.html - this is the HTML for the web bot interface. Place this inside a Templates folder underneath the app’s primary working directory.
requirements.txt - this is the file that is used to maintain the list of required libraries. This will be most important when creating a Web App deployment for Azure App Services.
webchatconfig.json - this is the configuration file that is called from WebChatBot.py. This will need to be modified with your specific Azure Open AI information such as: model name, API key, API base URL, and GTP type.
Don’t hate me, but I did get a little silly with my implementation and insert a reference Clippy (image) and it has adapted my own persona. You can modify the index.html file to include your own images and your own self-attribution.
Make sure you have Python installed. If you’re running a Windows system (like me), you can find the most current version of Python here: https://www.python.org/downloads/
This solution uses the Flask framework and the Azure OpenAI SDK. After you install Python, you can make sure you have both Flask and Azure OpenAI SDK installed by running the following install commands in a CMD window.
pip install Flask
pip install openai
Once all the requirements are in place, you run the app using the following command in a CMD window:
python WebAppChatbot.py
The app will instruct you to open a browser window to http://127.0.0.1:5000/ to use your new Chatbot.
NOTE: Unfortunately, this type of implementation doesn’t yet support ChatGPT-4, so you’ll see that the webchatconfig.json file references gpt-35-turbo. So, yes, the current version of this webchat does support only GPT-35. I plan to convert it to ChatGPT-4 as soon as its available so stay tuned for that.
Next Steps
The output and code is simple, so I’ll be working on this periodically for look, feel, and quality and posting updates to the GitHub repo. I also want it to run in a modal window so it can sit alongside other work. But what exists today is enough for now. It gives me great data for security monitoring and I’ll be using this in demos during a conference I’m attending soon.
Lastly, I plan to push this implementation to an Azure App Service Web App soon and will walk through how to do that in a future blog post. This is all in an effort to take security monitoring to the next level as many of our customers and partners will be doing.
Have fun with it. If you take what I’ve developed and make something super excellent, let me know.
P.S. I was showing my 15-year-old daughter this web bot, talking about how I’ve trained the data to make it more security-focused to help showcase our security products. She wants to be a developer someday. She thought it was neat, but her first question?
Lastly — Did I mention I’m not a developer?? I know the code is probably horrible and I would welcome any tips and help. My goal is not to develop something I have to support, but only to help get your creative juices flowing. However, it would be nice if the code was presentable.
[Want to discuss this further? Hit me up on Twitter or LinkedIn]
[Subscribe to the RSS feed for this blog]
[Subscribe to the Weekly Microsoft Sentinel Newsletter]
[Subscribe to the Weekly Microsoft Defender Newsletter]
[Learn KQL with the Must Learn KQL series and book]