Exchange 2016 & NSX Edge load balancing

This is going to be a quick and dirty post with not much details. Sorry. Spent some time figuring this out today and I wanted to put it here as a reference for anyone else. 

At work we needed to setup two Exchange 2016 servers behind an NSX Edge load balancer. We wanted to capture the source IP too so this meant we had to use the load balancer in transparent mode. A colleague had set it up already but the Exchange servers weren’t seeing the source IP so I took a look to see what was missing. I had to make two changes primarily to get it working. 

First: the Edge had two interfaces assigned to it. One for HA, another that connected to the backend servers. The VIP for the Edge was also in the second subnet. (This needn’t always be the case. I think a usual scenario is for the VIP to be on a different subnet so YMMV). For the Edge to pass on the source IP to the Exchange servers I knew I had to set it up in a transparent mode. This means the Edge passes on any packets it gets off to the appropriate backend server. It does not change the IP in the packet to be itself, so the backend server see the correct source IP. The problem with this though is that the backend server will then send this packet to its default router and from there to the source IP … which we don’t want. We want the source to only see the Edge VIP always and this means we need the return traffic to come back to the Edge, and the Edge will do a NAT to change the backend server IP to be the VIP. 

(This blog post might be a good starting point to read more on the above).

For an Edge to be setup in this fashion we have to set the Edge as the default gateway for the backend servers. (Which is straight-forward – just change the default gateway in the server OS). But for the Edge to then act also as a router for the backend servers we have to tell it that such and such IP is on an internal network and it is ok to do routing on that. In our case the HA network had been set as of type Internal, but the network that was connecting to the backend servers was incorrectly set as Uplink. I changed its type to Internal. This is important as this is what tells the Edge that it can now route any traffic coming to the IP addresses defined on that interface. 

This done, I went to the two backend servers and changed their default gateway to be the IP defined on the above Internal interface (vNIC0 in my case). I tested connectivity and also did a trace route to confirm it is going via the Edge. Great!

Second: By default an Edge load balancer is in L7 mode. When in L7 mode the Edge doesn’t forward on the packets it receives. It initiates a new connection to the Exchange servers, so the source IP is itself and defeats what we are trying to do. L7 mode is useful if we are doing any L7 manipulation such as SSL termination, cookie based persistence, URL rewriting, header insertion, etc. We are not doing anything like that here so I changed the mode to L4. This is done by enabling acceleration. 

Did the same in the virtual server too (this option is only available after enabling it in the above section).

Lastly, I enabled transparent mode on the pool.

That’s all!