12 min to read.
Abstract
Below conversation may come as a surprise for some
organizations when they onboard on Azure Firewall – [Click on image to get
better view]
Azure Firewall randomly selects attached
public IPs for outbound SNAT connections.
Many times for outbound connections, organizations
prefer to use static public IP for Outbound SNAT connections. This helps a lot
in whitelisting at organization customer’s / partner’s end. As of today Azure
Firewall do not offer this capability.
This article talks about; how can you make sure that
traffic leaving Azure Firewall uses a static public outbound IP addresses or
range of static public IP addresses.
The solution proposed below is being used in many large
enterprises today successfully, for providing static outbound IP to Azure
Firewall.
Let’s go!
Understanding inbound and outbound connections
In below diagram I have one Azure VNET in which I have
deployed one windows server and Azure Firewall instance in dedicated subnet.
Let us quickly understand what is inbound and outbound traffic flow.
The Azure VM do not have any public IP assigned to it.
If I host a sample application on top of this Azure VM then public facing
endpoint is public IP of Azure Firewall.
Inbound means traffic originating from
internet and reaching to Azure VM through DNAT operation of Azure Firewall.
Refer below diagram – [click on image to get better view].
Note – Remember, for
the inbound connection there will be response sent. This is response traffic
and not outbound traffic.
Outbound means traffic generated within
Azure environment and going out to internet. Refer below diagram
– [click on image to get better view].
Problem Statement
No Outbound Static IP
Now that inbound and outbound traffic is clear; in above
diagram Partner DC may ask for public IP of your Azure environment to whitelist
in their firewalls. So in that case we will need static public IP/ range of
public Ips assigned to firewall.
This way outbound traffic generated with Azure [in our
case inside Azure VM] to partner DC will be SNAT to static public IP.
However as stated above; Azure Firewall randomly picks
up any public IP assigned to it for outbound SNAT. This proposes challenges to
customers.
Limited SNAT ports
Another important point to consider is about SNAT ports.
When traffic goes out from firewall, a port will be used to send the traffic
outbound. These number of ports are always per public IP assigned to Azure
Firewall. Azure Firewall offers 2048 ports per public IP assigned. This might
be a lower number based on application nature.
Whenever one outbound connection is made, one port will
used from Azure Firewall. The engaged port is not released until connection
operation is completed. At the same time if Azure Firewall needs to make
another outbound connection then another port will be used.
This means, at any given point, one instance of Azure
Firewall service with one public IP attached, can make maximum 2048 outbound
concurrent connections.
Azure Firewall support max 250 public Ips. So total SNAT
ports available on Azure Firewall – 250*2048 = 512,000.
However, 250 public IP is still a big number. You do not
attach so many public Ips to Firewall instance out of the blue. You generally
attach per application. Bringing 250 applications behind single Azure Firewall
instance may not be possible right away when you start Azure journey.
At the same time; your single application running behind
Azure firewall may easily need concurrent a million outbound connections.
Example, WebSocket based chat application, mobile app communication using
websockets.
Therefore we need a way by which we can scale SNAT ports
as well for outbound connections passing through Azure Firewall.
Verifying Random IP Behavior
I have below construct in my Azure Subscription. [Click
on the image to get better view] –
1. Created
VNET with 3 subnets – Firewall Subnet, Web VM subnet, Jump VM Subnet
2. Attached
two public Ips to Azure Firewall.
3. Created
Web VM in Web Layer and did NOT attach any public IP.
4. Create
Route Table with below routes and attached to web layer subnet
a. If
destination is Jump Layer subnet – next hop VNET
b. If
destination is internet [0.0.0.0/0] – next hop AzureFirewall IP.
5. Created
Jump VM WITH Public IP; just to take RDP to web VM over its private ip.
Here are the screenshots of Azure Firewall IP configurations and UDR on Web Layer Subnet. [click to get better view].
I am going to add ifconfig.me domain inside Azure firewall to allow the outbound traffic. Therefore we need to add DNS settings on Azure Firewall. Below is the screenshot to add the same – [click to get better view].
We will be using ifconfig.me URL to get the IP. Therefore allow the same in Azure Firewall as shown below using application rules. [click to get better view].
Now when I run command Curl ifconfig.me/ip from putty of Web VM; we see the firewall IP as an output. This means because of UDR attached to web layer, internet traffic initiated from web vm is passing through Azure Firewall. Azure Firewall then sending to internet by SNAT [Source NAT] to its own one of the public IP.
From below screenshot you can clearly see that any random IP out of two public Ips attached
to Azure Firewall, is being used while outbound traffic is SNAT from Azure Firewall.
[click to get better view].
From above screenshot it is visible that out of the two public Ips random IP is being chosen for outbound access.
Here is the Deal…NAT Gateway!
Microsoft Azure releases Virtual Network NAT service
that simplifies the outbound-only internet connectivity for virtual networks. When
configured on a subnet, all outbound connectivity uses your specified static
public IP addresses.
Many times this service is also referred to as NAT
Gateway.
Per public IP attached to NAT gateway we get 64,000
outbound SNAT ports. One NAT gateway can have 16 public IP addresses attached.
So total outbound SNAT ports available with
a NAT gateway = 16 * 64,000 = 1,024,000.
So NAT Gateway is our solution to get Static
Public IP for outbound traffic flowing through Azure Firewall.
Create NAT gateway and associate to subnet of Azure
Firewall. Assign a public IP to NAT gateway. So as shown in the below diagram,
even if the Azure Firewall has 2 public IP addresses; traffic originating from
Web VM always take NAT Gateway associated public IP as outbound IP. [click to
get better view].
Create NAT GW and associate to AzureFirewallSubnet using below sample powershell commands. Make sure you replace Resource groups name, VNET Name and location as appropriate.
#
Create public IP addresses
New-AzPublicIpAddress -Name public-ip-1 -ResourceGroupName AzFwStaticIPRG -Sku Standard -AllocationMethod Static -Location 'Central India'
#
Create NAT gateway
$PublicIPAddress1 = Get-AzPublicIpAddress -Name public-ip-1 -ResourceGroupName AzFwStaticIPRG
New-AzNatGateway -Name firewall-nat -ResourceGroupName AzFwStaticIPRG -PublicIpAddress $PublicIPAddress1 -Location 'Central India' -Sku Standard
#
Associate NAT gateway to subnet
$virtualNetwork = Get-AzVirtualNetwork -Name MyVNET -ResourceGroupName AzFwStaticIPRG
$natGateway = Get-AzNatGateway -Name firewall-nat -ResourceGroupName AzFwStaticIPRG
$firewallSubnet = $virtualNetwork.subnets
| Where-Object -Property Name -eq AzureFirewallSubnet
$firewallSubnet.NatGateway
= $natGateway
$virtualNetwork | Set-AzVirtualNetwork
The Azure NAT Gateway I created has public IP address 52.x.x.20 and is visible on Azure portal as below – [click to get better view].
Similarly it is associated to AzureFirewallSubnet as shown below – [click to get better view].
Verifying Static Outbound IP Address
Logged in to webvm using SSH and ran curl ifconfig.me
command. We can see that it is showing Nat Gateway IP. Using NAT Gateway, for
outbound traffic passing through Azure Firewall NEVER selected random IP from
Azure firewall. [click to get better view].
Conclusion
Hope this article helped to overcome behavior of random
public IP selection of Azure Firewall for outbound traffic. NAT Gateway is
fully managed service and helps to configure static public outbound IP for outbound
traffic passing through Azure Firewall.
Happy Static IPs!
A humble request!
Internet is creating a lot of digital garbage. If you
feel this a quality blog and someone will definitely get benefited, don't
hesitate to hit share button present below. Your one share will save many
precious hours of a developer. Thank you.
Next
Related Posts
Proven Azure Architecture Patterns using PaloAlto NGFW and F5 DDoS and WAF
4
VNETs and transitive Routing using Azure firewall
Azure
Virtual Machines – real world frequently asked questions – not easily answered.
Exactly the information I was looking for.
ReplyDeleteProblem with NAT GW starts, when you need to have the same public IP for both inbound and outbound connections (because I assume you can attach it only to one of them).
For example you need your server to be hidden behind PIP 1.1.1.1 for incomming connections on ports 10101 and 10102, but at the same time this server has to be able to initiate connection in the opposite direction on port 10103.
For inbound, you always use Azure Firewall PIP. Then DNAT to backend destination.
DeleteNAT GW is not meant for inbound connection at all. Only outbound traffic flows through NAT GW. Not even response traffic.
I just wanted to thank you for this article. It has not only been helpful to me but also in explaining this situation to my company.
ReplyDeleteThis was much needed article, I was searching for this solution since two weeks
ReplyDeleteThis is good solution, however I am not able to deploy this in my infrastructure. In a region - I have a firewall and 3 Virtual Networks (each VNET has 2 subnets) Now when I am creating a NAT gateway and it can be added to subnets of one VNET at a time. It means I will have to create 3 NAT Gateways (1 in each VNET) but how to associate all those 3 to AzureFirewallSubnet because AzureFirewallSubnet is retained only in one VNET.
ReplyDeleteGood article! But let’s say the company wants to use their firewall to all their environments alpha, beta and prod. And all those environments need different outbound IP’s for white listening per least privileged access. We wouldn’t want alpha to be able to access prod data and so on. Would be great feature if we somehow could point address all traffic from 10.0.0.0/16 (alpha) use this public ip. If source is 10.1.0.0/16 (beta) use another public ip for outbound traffic. Is this possible? Can’t seem to find any information or configuration on this. BR
ReplyDeleteI am looking AT the same solution. It seems impossible without a FW Network Virtual Appliance ! If you find any solution with only azure components I will be very happy to get it.
DeleteThis is not a supported configuration if you are using Zone Redundancy with your Azure Firewall. This is a major failing of the Azure native solution, to such an extent that we have had to architect around this limitation.
DeleteYes, this configuration is not supported with Azure Firewall in Zone redundant deployment. As of today if firewall is no zone aware then NAT GW can be used on top of it to configure static outbound IPs.
Delete