Citrix XML Service headaches

Was setting up a Citrix XenDesktop environment in my test environment past few days and the Citrix XML service has been irritating me. There was no grand fix for the issue, but I spent quite a bit of time banging my head over it (and learnt some stuff along the way) so thought I’d make a post to put it all down.

Whenever I’d connect to the Storefront I get the following error:

I only get this error if you use the Receiver app by the way. If I try and connect via HTML5 you get no error at all. (So when in doubt, try with the Receiver always!)

I noticed that the “Citrix Delivery Service” event logs on the server had messages like these:

An SSL connection could not be established: You have not chosen to trust the issuer of the server’s security certificate, my-CA.. This message was reported from the Citrix XML Service at address https://mydeliverycontroller.mydomain/scripts/wpnbr.dll. The specified Citrix XML Service could not be contacted and has been temporarily removed from the list of active services.

I sorted that by changing all my certificates to SHA1. Turns out the default certificate signature algorithm from a Windows CA since 2008R2 is RSASSA-PSS, and Citrix doesn’t support RSASSA-PSS, so switching the CA to use SHA256 or SHA1 by creating a new CA certificate and server certificates is the way to go. In my case since this was a test lab and I didn’t want to encounter any more errors I went with SHA1.

I was mistaken however as I soon got the following error:

An SSL connection could not be established: An unclassified SSL error occurred.. This message was reported from the Citrix XML Service at address https://mydelivercontroller.mydomain/scripts/wpnbr.dll. The specified Citrix XML Service could not be contacted and has been temporarily removed from the list of active services.

This one had me stumped for a long time. I know all my certificates were proper, and they were bound correctly to IIS, so what was this error about? Moreover it didn’t give much details, and there were not many forum or blog post hits either. Everything looked fine – so what the heck?! If I told the Storefront to communicate with the Delivery Controllers over HTTP instead of HTTPS, things worked. So clearly the problem was with HTTPS.

I was able to visit the XML Service URL too with no certificate errors.

Here’s an excellent post on the Citrix XML Service. The important thing to note is that if the IIS role is already present on the server when the Citrix XML Service is being installed, it integrates with IIS; whereas if the IIS role is not present the Citrix XML Service operates in standalone mode. During my install I didn’t have IIS, but since IIS got installed as part of the install I thought Citrix XML Service must be running integrated – but it does not. In my case Citrix XML Service is running standalone.

Anyways, not a good idea to integrate the Citrix XML Service with IIS, so I am going to leave mine standalone. Here’s a Citrix KB article on how to integrate though. Also, for my own info – the registry keys for the Citrix XML Service are under HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\DesktopServer. Apart from the default ones that are present one can also add two DWORD keys XmlServicesEnableNonSsl and XmlServicesEnableSsl to manipulate whether the Citrix XML Service accepts HTTP and HTTPS traffic. By default both keys are not present and have a value of 1, but changing these to 0 will disable HTTP or HTTPS.

Back to HTTPS and the Citrix XML Service. Since it is not integrated in my case, I should follow the SSL instructions for standalone mode. Roughly:

  1. Install the server certificate as usual.
  2. Note the certificate thumbprint.
  3. Find the Citrix Broker Service GUID. Do this via wmic product list (hat tip to this blog post for the latter idea; alternatively the Citrix article shows how to do this via registry).
  4. Use netsh to bind the two.

In my case the command would be something like this:

I didn’t need to do this as my correct certificate was already bound. It was bound to IIS  I guess (the appid wasn’t that of the Citrix Broker Service) so I double checked by removing binding and creating a new one specifically for the Citrix Broker Service. Still no luck!

If I were running Server 2016 there’s some additional steps to follow. But I am running Server 2012 R2.

My setup was such that I had two Delivery Controllers and one of them had the Storefront. It didn’t make a difference which Delivery Controller I chose to add to the Storefront – it never worked. At the same time, switching to HTTP instead of HTTPS always worked. I had no ideas. I posted to the Citrix forums too but only got one reply. Frustrating!

On a whim I installed Storefront on the second Delivery Controller server to see if that works. And it did. The Storefront on that server was able to talk to either Delivery Controllers with no issue. So the issue wasn’t with the Delivery Controllers. For some reason I had always thought the issue was with the Delivery Controllers (I guess because the error message was from the Citrix XML Server/ Citrix Broker Service and that is a part of the Delivery Controller) but now I realized it was to do with the Storefront. And specifically that particular server. I uninstalled and re-installed the Storefront but that didn’t make a difference.

My next suspect was certificates so I compared the trusted root CAs between the broken server and the working server. I found that the broken server had some of my older root CA certificates (remember I had switched my DC/ CA from SHA256 to SHA1) so maybe that was causing an issue? It also had an extra DigiCert certificate. I removed all these and tried again – and voila! it worked!

I am pretty sure I had manually removed all these older DC/ CA certs, so I am not entirely convinced that is the cause. But it sounds plausible and maybe they came back even though I removed.

Update: I hit upon this error again after I stupidly went and renewed my root CA cert (which is my Domain Controller). Stupid, coz I was doing it just for the heck of it (it’s my test lab after all!) but that broke the certs on the Delivery Controllers/ Store Fronts and I began getting these errors again. As a work around I went have deleted the new certs from the local stores of these (Trusted Root CA and also Intermediate CA) . Am sure it will sync in again, so long term I better regenerate my certs or just turn off SSL internally. Most likely the latter as I am lazy. :p