Azure API Management Disaster Recovery in Standard – Part 2 – Traffic Manager with custom domain and way forward

Abstract

In the first part - https://sanganakauthority.blogspot.com/2019/08/azure-api-management-disaster-recovery.html we have configured below things –

1.      Created required resources in primary and secondary region
2.      Configured the Azure AD application and provided necessary permissions
3.      Configured the backup and restore functionality for Azure API Management and automated the same using Azure Logic Apps. This is crux of our DR solution of API Management in Standard sku.

In this part we will configure below things –

1.      Traffic manager configuration for distributing load to primary always and to secondary in case disaster.
2.      Configure the status page of API Management in traffic manager
3.      Configure custom domain and certificate for API Management and traffic manager
4.      Test the failover by downgrading one of backend APIs and mocking the response
5.      Benefits of approach and conclusion

Traffic Manager configuration

Create traffic manager as per the link described in secondary region resource group – https://docs.microsoft.com/en-us/azure/traffic-manager/quickstart-create-traffic-manager-profile#create-a-traffic-manager-profile .

Provide values as shown below –

Note the URL of API Management standard instances from primary and secondary instances as shown below  and highlighted by green line –

We will use these URLs as an “External endpoint” in traffic manager profile. Configure the traffic manager endpoint as shown below. Secondary API Management as priority 2 and primary API Management as priority 1.

Configure Health Status Page of API Management in Traffic Manager

Azure API Management Disaster Recovery in Standard – Part 1 – Backup and restore configuration

Abstract

This blog post explains step by step configuration for achieving Azure API Management DR in Standard sku. This bog post is divided in 2 parts series. Links to all parts is as below –

Part 1 – Backup and restore configuration for API Management

Part 2 – Traffic manager and custom domain configuration,testing the DR of Azure API Management

Being intelligent human being you can directly go to any of the part; but I would recommend to go ahead sequentially from Part 1 to Part 2 of the blog post. This will ensure that there are no mistakes. Ask any questions in comments. If I see that question asked is already answered in any of the part of this series; please don’t expect the answer to the question asked.

Till now we have seen two important blog post aboutAzure API Management custom Role Based Access Control and Need of API Gateway in modern architectures as concept.

This blog post takes another deep dive discussion in API Management DR configuration.

Azure API Management Disaster Recovery in Standard – the need?

Azure API Management as of today offered in various pricing sku with various features –1.    Consumption

2.      Developer

3.      Basic

4.      Standard

5.      Premium.

When you onboard API Management as a part of your API story for all of your Enterprise applications; this component becomes mission critical. As a best practice we should also configure Disaster Recovery (DR – multi region deployment) for API Management.

API Management DR is by default provided as a service in Premium sku. However at the same time the capacity or workload support by premium sku is massive. Therefore premium sku also comes with premium price. Sometimes enterprise needs are observed to get satisfied by BASIC or STANDARD pricing tier. As of today as a bench marking of API Management with STANDARD tier support approx. 2500 req/ seconds per unit. STANDARD tier can have up to 4 units. Means total number of requests that can be served by STANDARD API Management can go up to 10000 req/ sec. This is massive scale and may satisfy most of the Enterprise requirements. However STANDARD sku do not provide Disaster Recovery [DR] pre-configured. This may propose bigger challenge for organizations who are betting big on Azure API Management.

This document provides step by step document for configuration of DR for API Management STANDARD sku along with architecture. Hope this document will help to address the concern of API Management DR in Standard/ Basic mode.

List of Azure service used in the API Management DR with Standard

As a part of the solution we will use below services in Azure to implement API Management DR in standard mode.

1. 2 Logic apps – for automation of backup from primary region and restore in secondary region of API Management
2. API Management in two regions – Central India(Primary - CI) and South India(Secondary - SI)
3. Traffic Manager – to distribute the load in case of DR situation
4. Storage account – for storing backup information of API Management primary
5. Azure AD application – to allow logic app provide authentication mechanism to make changes in API Management (through ServicePrincipal concept).

High Level Steps

The gist of this article is based on backup and restore feature of “API Management” management REST APIs. The REST API documentation is present below –

Backup - https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-01-01/apimanagementservice/backup

Restore - https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-01-01/apimanagementservice/restore

These APIs allow backup and restore operation for entire Azure API Management instance and it is crux of our solution. Refer to high level steps for the solution below.

1. Create two API Management in Standard mode. One in primary [in our case Central India] and other in secondary [in our case South India].
2.  Configure APIs and related backend in API Management primary. As you API Management standard is not part of VNET, the backend APIs will require public IP to get imported in API Management.
3. Create Azure AD Application to have necessary permission to make changes in API Management instances in cross region.
4. Allow rights on resource groups of both API Management in two regions.
5. Create two logic apps. One to create backup of primary API Management. Second will be used to restore the same backup in secondary region API Management.
6. Create traffic manager and add API Management IP address/ domains names as endpoint profiles. Configure Traffic manager in Failover/ Priority mode.
7. Configure status page of API Management in Traffic manager to know health status of API Management for traffic manager so that it can switch to secondary region in case of DR situation.

Architecture for API Management DR in Standard Sku

1. Logic app uses timer orchestration step to automatically trigger itself. As a next step same logic app sends service principal details to Azure AD and then retrieves the authentication token.
2. Invoke API Management rest API with operation as backup.
3. The operation starts copying data to storage account in other region. Depending on the size and number of APIs being used in Azure API Management the backup operation takes approximately 1-2 hours [this may change based on the data present in your API Management instance. Don’t quote me if it takes more/less time than I mentioned].
4. Post successful copy operation of backup; another logic app is triggered which restores the backup data in another API Management instance created in secondary region.
5. Post restore operation both API Management instances are now exactly identical. For example, in secondary API Management all policies, subscription key, APIs etc. will be same as primary API Management.
6. The request from end user/ application is received to traffic manager. Traffic manager is configured in failover mode with primary API Management as active and secondary API Management as passive.
7. Traffic manager passes on the incoming request to active API Management instance.
8. The health status page of both API Management instances /status023456789abcdef keeps sending 200 response to traffic manager. In case if primary status page sends non-200 response then Traffic manager will route the traffic to secondary API Management automatically.

Step by step execution

Azure API Management – Restrict users to specific products using Azure custom roles based RBAC

Abstract

Post you understand the importanceof API Gateway in modern architectures it is time now to go deep dive in Azure API Management.

Few of my customers are already on this journey of using Azure API Management. I have observed a pattern that if you use Azure API Management then in few moths it automatically becomes a central point of all your deployments and API Management will be very mission critical to your entire business. As you grow your footprint on API Management there can be many unforeseen challenges you face and hence it is better to plan in advance.

With one of my customer they are facing lot of challenges now with respect to governance and access control of entire API Management instance in their azure subscription. Let us understand the problem they are facing and how it can be solved as of today.

Let’s go!

Problem Statement

Basically customer is having around 10 applications [with approx. 3-4 folks in a team/ application] whose APIs to be published into Azure API Management. In such cases I always recommend to have following structure as a best practice which can help to better manage your APIs in Azure API Management.

[I am using fictitious application names below – ]

#	Application Name	Azure API Management - Product recommendation
1	HR	Create a product named as “HR”
2	Finance	Create a product named as “Finance”
3	And so on…

So essentially you should have at least one dedicated product in Azure API Management per application. In case of need you can have two or more products per applications, but at least one azure API Management product per application is a must. Do not combine two or more applications into single Azure API Management Product.

So now they have around 10 products and total 50+ people team working on the same instance of Azure API Management. All of the team members wanted to publish APIs in products and this where problem starts.

Human nature is of all about building experiences. And those experience you gain by making mistakes. Making mistakes is not an issue but it is bad that you don’t work for solving those issues.

Team were accidentally modifying/ deleting/ changing/ creating products and APIs which they were not supposed to touch upon. Some even re-generated subscription key for APIs or changed the VNET configuration for API Management without letting others know. Such changes never bring Azure API Management down; but this led to some horrific abrupt downtimes of their APIs accessibility through API Management. This was happening because of lack of governance and access control issues. Let us understand more.

Current available access control RBAC options in Azure API Management and what do we need?

The structure of API Management looks like below –

There are two types of users an API Management has at high level as shown below –

In order to enable for all the team members to publish and import API in Azure API Management they will have to be given access as “Azure API Management Service Contributor”. However providing this access to any team member provides full access over API Management. There are other roles defined however few of them are not yet released and there is no role that can help to restrict access to a particular product in Azure API Management. Refer to current existent roles in Azure API Management - https://docs.microsoft.com/en-us/azure/api-management/api-management-role-based-access-control.

So in summary we want to achieve below -

1.             User from HR app should have access to only HR product in Azure API Management for any modifications. The other products like Finance and so on should only be read only.
2.           User from Finance app should have access to only Finance product in Azure API Management for any modifications. The other products like HR and so on should only be read only.
3.         Read only access for entire API Management.
4.        They should not be able to delete any product/ API or anything from Azure API Management.

Define custom role for Azure API Management using JSON and PowerShell

I have created one API Management instance in my subscription. Added few APIs. By default you get 2 products – Starter, Unlimited. These are default and I am not touching them. I create 2 more products namely “HR” and “Finance” as shown below –

The options “Users, Subscriptions and Groups” in your API Management instance are basically for Developers who wants to consume the API. It is no way related to users who want to publish APIs inside a product. So don’t confuse with these options with what we plan to achieve. Refer to below screenshot –

It is always best way to start with existing definition of Azure API Management RBAC role and then modify it to suit to your requirements. Looking at the requirements I have stated above it makes sense to download the existing JSON of “API Management Reader” role and then modify it. To download the Reader role details we use Azure PowerShell. Here I assume that you already have Azure PowerShell installed and you are all set.

First run the login azure account command as shown below –

Login-AzAccount

After successful login now it is time to download the API Management Service Reader Role JSON file. Run below command to download the Azure API Management Reader Role details –

Get-AzRoleDefinition -Name "API Management Service Reader Role" | ConvertTo-Json | Out-File "Your Local machine path\APIMProductAdminCustomRole.json"

In above command make sure that you replace the Path with your choice to create the JSON file. Once you have the JSON file created; open in VSCode or Visual studio or any of your favorite editor. As the new role we are creating is custom so make IsCustom field as true. Similarly the current Id field is from Reader role and for our custom role it has to be newly generated. So simply remove it from JSON. When we create a custom role it will get automatically create.

In Action section we don’t need our role to make any deployments in resource group therefore property 
Microsoft.Resources/deployments/* is unnecessary. So remove it. At the same time we need our custom role based user to create a support case and should be able to read/ view other services/ resources from the resource group. Therefore keep all the options as shown below -

"Microsoft.ApiManagement/service/*/read",

"Microsoft.ApiManagement/service/read",

"Microsoft.Authorization/*/read",

"Microsoft.Insights/alertRules/*",

"Microsoft.ResourceHealth/availabilityStatuses/read",

"Microsoft.Resources/subscriptions/resourceGroups/read",

"Microsoft.Support/*"

This completes Action part. We need more focus on NoActions part of the JSON and this is crux of our custom role. Therefore add below options and their description is also provided in comments.

"Microsoft.ApiManagement/service/users/keys/read",

"Microsoft.ApiManagement/service/delete", //no delete of API Management instance is allowed

"Microsoft.ApiManagement/service/write", //new API Management can't be created

"Microsoft.ApiManagement/service/products/delete", //product can't be deleted

"Microsoft.Resources/deployments/*" //create and manage resource group deployments

So in summary we are not allowing this custom role to perform –

1.          Any deployments
2.          Any type of delete operation on Azure API Management
3.          Creating new API Management instance
4.          Delete any Azure API Management products
5.          Reading of the secret keys is also not allowed.

Now we need to “scope” these permissions at Product level of Azure API Management. We have created 2 products – HR and Finance in API Management. Therefore we want HR team only to update/ read HR product, can view Finance product but should be able to perform any update/ delete operation on Finance product. So we need to Role based access control for Azure APi Management at product level –

1.    .      Role named as - API Management Service Product Admin - HR
2.           Role named as - API Management Service Product Admin – Finance

Essentially we need to create RBAC custom role dedicated for each of the Azure API Management product.

So for this JSON add name at the top as shown above for HR. Now we need to assign this custom role to only HR product under Azure API Management. Therefore let us retrieve the id of HR product from Azure API Management.

Run below commands to get the ID of a product under Azure API Management –

#get the id of product in API Management

$apimContext = New-AzApiManagementContext -ResourceGroupName “Resource group underwhich API Management is present” -ServiceName "Your API Management instance name"

Get-AzApiManagementProduct -Context $apimContext -Title YourProductName

The output is shown as below for my HR product –

Add this highlighted ID in AssignableScopes section inside your custom role JSON file as shown below –

This completes building custom role JSON file for Azure API Management to provide specific product level update access; and restrict delete/ update/ write operations on other products inside Azure API Management.

Now we need to execute this JSON using PowerShell further to create the custom role inside Azure subscription.

Create Custom Role using PowerShell

Now our final JSON document can be used to create actual custom role in Azure API Management. Therefore run below commands in PowerShell –

#to create new role for the first time

New-AzRoleDefinition -InputFile  "Your local path of choice \ APIMProductAdminCustomRole.json"

This command will automatically generate Id in the output window. We need to copy and paste this Id inside the JSON. So subsequent upgrades against the same role can be achieved. The Id generated is as below –

Copy paste this id as below in your JSON file -

To verify the role has ben created successfully run below command –

#to retrive role definition ID in powershell object

$role = Get-AzRoleDefinition "This name is of your role as highlighted green in above screenshot"

$role

This completes the custom role creation dedicated for Azure API Management product named as HR. So depending on the other products you can create many roles dedicated for each of the product.

Assign custom role to User

Now that we have custom role already created we need to assign it to a user. The user can be part of your Azure AD in which your subscription is present or it can be a Microsoft account [live id/ Hotmail id etc].

For simplicity I am going to use Microsoft account here. The process for Azure AD based user remains the same. For assigning the Azure custom role to user we need its object ID and SignInName [also known as User Principal Name]. Let us first retrieve the same.

Get-AzADUser -StartsWith "YourUserName"

Below screenshot shows which all records to be captured. Green box is SignInName and Red box is Object Id or the user –

Then retrieve the complete Id of your Azure API Management instance which will be used while assigning the access to user.

$apim = Get-AzApiManagement -ResourceGroupName YourResourceGroupName -Name "Your Azure API Management instance name"

$apim.Id

Record the id displayed in above step.

#assign role at resource scope level means at API Management level directly

New-AzRoleAssignment -ObjectId "USerObject Id" -RoleDefinitionName "Reader" -ResourceName "YourAPIManagement Name" -ResourceType Microsoft.ApiManagement/service -ResourceGroupName "YourResourceGroupName"

New-AzRoleAssignment  -SignInName "User SignInName" -RoleDefinitionName "Name of custom role present in your JSON file" -Scope "Id of product retrived in earlier steps."

Now this role assignment is not displayed in “Access Control” option on the portal. As of today I guess custom roles created by you will not be visible if they are next level than actual resource. For example Product is present inside Azure API Management. So custom role created at API Management level may be visible but role created at inner components scope like product is not visible on Azure portal as of today.

So verify if the role assignment is completed or not by running below command –

#to check level of access for a user on API Management use below powershell - custom roles will not be visible on the portal

Get-AzRoleAssignment -ObjectId ObjectId of your user

Verifying the role assignment at product level of Azure API Management

Login to azure portal with the user who has access to HR [or your respective product] product. After login API Management instance should be visible. When we open the products all products should also be visible. However if we try to delete/ update any other products than assigned one then we should get error similar to below –

Conclusion

And that’s it! Hope this article has given you how can you control Role based access control in Azure API Management at deep level.

Download the final PowerShell code and JSON I used in above article from Github here –  https://github.com/kunalchandratre1/AzureAPIManagementCustomRoleAtProductLevel

Happy RBACing!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefitted, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.