NetScaler/ Exchange RPC – TCP syn sent, reset received

At work one of my colleagues is setting up NetScalers as load balancers for our new Exchange environment. He is replicating the existing setup but found that the RPC 60001 & 60002 Service Groups on the NetScalers were being marked as down. Curious, I took a look.

After SSH-ing into the NetScaler I could see the following via show serviceGroup <serviceGroupName>:

My colleague too had seen this and pointed me to a good blog post from Citrix on what the reset codes mean. That blog post is a good one (that’s why I am linking it here, as a reference to myself) but I don’t think he was looking at the trace via a NetScaler trace so we had no idea of the codes. (Speaking of which, here’s a good post on NetScaler and Wireshark. Here’s a KB article on how to collect traces from NetScaler. And here’s a KB article on how to collect traces from the CLI. Whilst I have briefly read them, I haven’t tried them out currently). 

Back to the issue at hand. I could see that the individual servers (Exchange 2010 Client Access) were up on RPC 135 and HTTPS, but only RPC 60001 & 60002 were down. I decided to do a portQry against a server in the older environment and compare against the new. Here’s the relevant bits from an older server:

As expected, something is listening on ports 60001 and 60002. When I tried the same against the new server, however, there was nothing listening on either of these ports. I searched the output based on the UUIDs and found the port numbers were different:

So that’s why the NetScalers were getting a reset. Nothing was listening on those ports! Solution is simple. Configure these RPC ports as static.

That’s all! :)