8 min to read.

Abstract

Azure Open AI adoption for application innovation is in full speed. With the same speed security teams are analyzing and doing VAPT on Azure Open AI endpoints. Recently a customer’s security team gave big list of 30+ vulnerabilities for their Azure Open AI endpoint.

So obvious question from application team “what can we do for security of Azure Open AI?”

This article step by step guide is for using Azure Front Door Web Application Firewall (WAF) with Azure Open AI.

The Challenge

In my opinion performing VAPT on direct open AI endpoint and calling it unsecure is incorrect approach. If an application team develops and host APIs on say Azure App Service/ AKS/ ARO/ Container Apps; it will also have similar vulnerabilities. Because these are plain APIs and not security appliances. We need to use Security appliances in front of APIs to secure them.

This is similar type of scenario when you create Azure Open AI service in your Azure subscription. Therefore we need to address below security aspects for Azure Open AI endpoint –

       1.      OWASP common vulnerabilities protection
2.      Azure Open AI endpoint should not be hit directly from end client applications like web app/ mobile app.
3.      If possible, remove public access to Open AI endpoint completely.

Solution

The standard approach to protect Azure Open AI APIs will be to use Web Application Firewall (WAF).

WAF on Azure Native technology offering is either Application Gateway or Azure Front Door. I will use Azure Front Door in this article.

Lets start.

Create Azure Open AI Service

Please don’t ask “how do I get access to Open AI in my Azure subscription?”. It is not in my control and I can’t even influence it. Follow the general procedure mentioned here.

Once you have access to Azure Open AI follow the below procedure to create it.

How-to:Create and deploy an Azure OpenAI Service resource - Azure OpenAI | MicrosoftLearn

From this link make sure that you choose “option 1: Allow all networks”. Don’t follow “specific networks only” and “disable network access” scenario for now.

Now Azure Open AI service instance is created. Next step will be to create deployment. Click on “Create New Deployment -> Add details as shown with model selected as gpt-35-turbo. Click on Create to complete model deployment.

Once model is created, select it and click on “Open in Playground”. Then click on “View Code” under “Chat Session” section. A pop up will appear showing Azure Open AI endpoint, and an access key as shown below –

Lets try Azure Open AI endpoint in Postman and get the output. To run in postman I used below details –

Body added in postman is as follows –

{

"messages": [

{

"role": "system",

"content": "You are an AI assistant that helps people find information."

{

"role": "user",

"content": "What is Azure OpenAI?"

}

]

}

Rest of the setting with header in Postman as below. As you can see the outcome is 200 successful response.

Here we have used Azure Open AI endpoint directly in postman. As a next step lets create Azure Front Door and configure Azure Open AI as a backend to Azure Front Door.

Create Azure Front Door Service

On Azure portal click on Create -> In Search box type FrontDoor and Select FrontDoor Service creation option. Then on offerings page select options “Azure Front Door” and “Custom Create” as shown below – [click on image to get better view].

On Profile page under Basic tab select tier as “premium” and Resource group, service name as per your choice. Refer details as below. I could have chosen Standard tier however later in the future I am going to show AFD + Azure OpenAI with private endpoint which will be supported in Premium tier of AFD. [click on image to get better view].

Keep Secrets tab empty for now. On “Endpoint” tab click on “ Add an endpoint” button. Provide name as you wish. The click on Add. Refer to below screenshot – [click on image to get better view].

Then click on “Add a route” option in “Endpoint” tab. Add name as “myRoute”. Then click on “Add a new Origin Group”. Name it as “myOriginGroup”. Then click on “Add a Origin” button as shown below –

On the add Origin screen, I selected origin type as “custom” as Azure OpenAI does not appear inside the Azure front door origin types list. As of now I have not enabled the private endpoint for Azure Open AI. Refer to the screenshot below for rest of the inputs. Then click on “add” to finalize on Origin creation.

You will be redirected back to Origin group screen. Here the screen asks about enabling health probe. In Azure Front Door Health Probes can’t work with Authentication, however Azure Open AI endpoint API need API Key header for authentication. Also my intent is to show how we can WAF protection for Open AI. There are other ways by which you can check Azure Open AI endpoint health status. Therefore here I will disable the Health Probe. Click on Add button to finish Origin group creation.

This will bring back to add route screen. Keep Origin Path blank, Forwarding protocol to “match incoming requests”, Caching option is unchecked. Then click on add to finish route adding.

On the same page click on “Add Policy” button. Provide the name as “myPolicy”, select Azure OpenAI domain listed under “Domains” dropdown. Under WAF Policy option, click on Create New and name it as MyWAFPolicy. Check the option “add bot protection”. Finally click on Save to complete WAF policy creation.

Then click on “review and Create” button to finish Azure front door service creation process.

Output

Now we have Azure Front Door WAF ready with Azure Open AI endpoint as backend. Lets try to access Open AI endpoint using Azure Front Door URL via Postman.

My original Azure Open AI Endpoint is - https://kunalgenai.openai.azure.com/openai/deployments/kunaldeploy1/chat/completions?api-version=2023-03-15-preview

With Azure Front Door it will be - https://kunalopenaiafdep-hgc2ecbybxeranac.z01.azurefd.net/openai/deployments/kunaldeploy1/chat/completions?api-version=2023-03-15-preview

Try the Azure Front Door URL in postman, with same body, same headers, same api key and I see that result is successful. Refer below screenshot –

I have not tested with Azure Open AI with private endpoint. Please test it and add your experience in comments.

Conclusion

Hope this article helped you to secure Azure Open AI endpoint using Azure Front Door (AFD) (WAF).

Happy Generative AI..ing!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

AzureVirtual Machines should be connected to allowed multiple virtual networks.

Start stop multiple Azure VMs on schedule and save cost!

Export Azure VMs to CSV!

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!