Contact

Subscribe via Email

Subscribe via RSS/JSON

Categories

Recent Posts

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

The Citrix servers do not trust the server. This message was reported from the XML Service at address …

If you are able to login to your Citrix Storefront and get a list of application, but when launching something you get an error –

And checking the “Citrix Delivery Services” logs on your Storefront gives errors such as these –

  • The Citrix servers do not trust the server. This message was reported from the XML Service at address http://xxx/scripts/wpnbr.dll [NFuseProtocol.TRequestAddress].
  • Failed to launch the resource 'xxxx' using the Citrix XML Service at address 'http://xxx/scripts/wpnbr.dll'. The XML service returned error: 'not-trusted'.

You have come to the right place. :)

This is because you probably have “Domain pass-through” authentication enabled on your Store and/ or the Receiver for Websites (note the latter: easy to miss out). When this is enabled and users visit the Storefront page, they don’t get the usual username password prompt. Instead they get an option to login with the Windows credentials.

The problem with this is that these Windows credentials are passed on to the Storefront server. The Storefront server is happy, gives the users a list of apps and desktops assigned to them, etc. But when the user clicks on something, it is the Citrix Receiver that comes into play and needs to pass on the credentials to the concerned XenApp or XenDesktop server. Citrix Receiver needs to be explicitly installed with the ability to do Single Sign-On (i.e. pass on Windows credentials) and if that’s not the case users will not be able to launch any app or desktop. (The command line to do such an install is: CitrixReceiver.exe /includeSSON). Once this is done an additional component called ssonsvr.exe will be present on the user machine, and that facilitates SSO.

For more details on Citrix Receiver and SSO check out this link. And for an official explanation of what I wrote above, check out this blog post.

Lastly, if you don’t want to do any of this, there is a work around. :)

Brief notes on NetScaler and Citrix StoreFront

I spent the last two days intermittently trying to set up NetScaler and Citrix StoreFront in my home lab. It was a mixed bag partly due to my nature of just jumping into something and figuring it out as I go along :) but compounded by the fact that while there’s a lot of documentation on the Internet they seem to be either outdated or don’t explain the big picture of what we are trying to achieve. (Or maybe I am just slow in picking it up – wouldn’t put that possibility aside!)

Anyways. Here’s a couple of stuff in no particular order. 

This PDF is the official documentation on setting up NetScaler with Citrix StoreFront. It’s a good one – lots of screenshots etc. Page 19 onwards seems to be outdated though with the latest version of NetScaler that I have – 11.1 53.11. 

When I started this exercise though I was on a much older version of NetScaler – 10.5 56.22. I started setting up the NetScaler as a gateway (e.g. http://ctxstorefront.myfqdn) to my internal StoreFront servers (http://data01.myfqdn and http://data02.myfqdn), and my steps were more or less along the lines of the PDF above which I discovered later. That wasn’t a success though coz when I’d connect to the NetScaler gateway it gave some errors (I forget what now). Digging into this I realized that the wizard also creates a load balanced virtual server on the NetScaler and that was showing as down. Dug into that a bit and found out that the underlying monitor probes to the two StoreFronts were failing. If I separately created services representing my StoreFronts and attach the same monitor (basically, a monitor of type STOREFRONT, with the correct StoreName etc) it fails. 

That was good to know coz I learnt a bit about the monitor probes. :) I found the following in my NetScaler logs:

That’s same as what the GUI was telling me but I was additionally able to find what error code was being generated (thanks to this KB article):

So it runs a Perl script basically. Nice. Went through that script and here’s the relevant section:

So it tries to access http(s)://<my delivery controller>/Citrix/<my Store>/discovery and gets a 200 error. Odd that it gets it, because if I try to probe that URL via cURL from NetScaler, I get a 404 error (which is sort of correct; I get a 403 error via IE and that’s the correct one I think):

Anyhoo, that stumped me for a while until I found forum/ blog post where they said upgrading to a newer version of NetScaler supposedly fixes this. So I went down that route, and yes, it helped. 

After upgrading though my UI changed. :)

The new UI is very different. But that’s good I think, coz it forced me think at least on what exactly am I trying to achieve here; and to take a step backward and understand what is going on. So here’s my understanding:

  • We have StoreFronts. :) That’s the thing you actually connect to. 
  • We can have a group of StoreFronts for High Availability. You can configure each of these independently or you can keep them in sync from the UI itself. 
  • For each StoreFront we need to specify a base URL.
    • For a single StoreFront it’s easy – http://data01 or whatever (where data01 is my StoreFront server name in this case).
    • But what about when I have a group of StoreFronts? If I am keeping them in sync via the UI itself, the base URL I define on one of them will be pushed out to all. So all my StoreFronts will have a base URL of http://data01 even though they might be called data01, data02, etc. That’s a problem coz all my clients will only connect to the first one as that’s where DNS will point clients to.
  • To avoid the above situation for multiple StoreFronts we need a common base URL which can be load balanced across all. A regular DNS round-robin situation won’t work coz we also need clients to stick on with whichever StoreFront they connect to, and also some form of monitoring to ignore StoreFronts that are down would be good to. So this is where the NetScaler first comes in!
  • Create a Virtual Server on the NetScaler that will load balance the various StoreFront services we define on it. Simple stuff – just HTTP or HTTPS services that are load balanced. Add the STOREFRONT type monitor. And a Persistence of type COOKIEINSERT. That’s all. (Oh, and add SSL certs etc if you are using HTTPS). 
    • I haven’t gone through this link but am putting it here as a reference to my future self – pretty sure I must have missed something when setting up things now. Also, that link goes into some scenarios such as where we do SSL termination at the NetScalers. 

What I realized as I thought about this is that this Virtual Server I create above is the main thing. Going by my URLs above, I should have http(s)://ctxstorefront.myqdn load balance among http(s)://data01 and http(s)://data02. Forget about the whole external access stuff for now. 

Once I do this and ensure things are working (they do), now I can think about external access. What does the NetScaler need to do there? It doesn’t need to do any load balancing coz that’s already setup; so all it needs to do is provide a VPN gateway sort of service for external clients to connect to! So that means a new Virtual IP on the NetScaler. Create this using the wizard in the “Integrate with Citrix Products” > “XenApp and XenDesktop”. This will basically create a Virtual Server in the NetScaler Gateway section (note: not in the Load Balancing section). As part of configuring, we have to point this Virtual Server to a StoreFront. Here use the Load Balanced StoreFront Virtual Server that we created on the NetScaler above – this is where it all ties in! 

I could use the same internal URL for the external access too I guess and use split DNS – not sure, because I do have to specify this on the StoreFronts and I haven’t tried/ thought of any side effects – but in my case I simply decided to go with a different URL for the external access. Specify that for the certificate and Virtual Server etc, add that to DNS, and now I can access the StoreFronts externally too via the NetScaler. 

If I were to go to the Virtual Server in the Gateway section there’s no obvious mapping from this one to the internal load balance Virtual Server. But there is. It’s in the policies section. That has a Published Application pointing to the load balanced URL (including the Store name (actually, there’s two policies – one for the Web another for the Receiver, to capture the different names)) so that’s how the traffic flow works. Users hit the gateway, the gateway does the authentication etc and checks its policies, finds one that offers uses the StoreFront published application at such and such URL, and it looks that up (it is with itself) and thus hits the load balanced Virtual Server, and so on … 

Finally! I have some idea of how this all ties in together. :)

[Aside] https://127.0.0.1 Citrix Store SSL discovery error

Due to a goof up on my part in my test lab, I was encountering this error. Found this forum post which helped me fix it; after which I realized the error was happening coz of a configuration error on my part. No point going into what my mistake was (in short – I have two servers that act as both StoreFront and Delivery Controller; the base URL of the StoreFront is one of the servers but I was also trying to access the StoreFront via the other and it worked but kept erroring (it worked coz the IIS website is there; and it errored coz I am not supposed to access it via that URL) so that was me being silly). 

Thank you Internet!

Time to setup NetScalers in my test environment so I can access the StoreFront via them. Which is what I should have done in the first place to load balance between these two servers. 

Refresher to myself StoreFront and Delivery Controller authentication

In a previous post I had written about the flow of communication between Citrix Storefront and Delivery Controllers during user authentication. Here’s some more based on a Citrix blog post I am reading. 

Here’s what I had written in my previous post:

There’s a couple of steps that happens when a user logs in to access a Citrix solution. First: the StoreFront authenticates the user against AD. Or if the user is accessing remotely, the NetScaler gateway authenticates the user and passes on details to the StoreFront. Then the StoreFront passes on this information to the Delivery Controller so the latter can give a list of resources the user has access to. The Delivery Controllers in turn authenticate the user AD. The Delivery Controller then sends a list of resources the user has access to, to the StoreFront, which sends this on to the user’s Citrix Receiver or Browser. This is when the user sees what is available to them, and can select what they want.

When the user selects what they want, this is information is passed on to the StoreFront, which then passes the info to the Delivery Controller – who then finds an appropriate host that can fulfill the requirement and sends this information to the StoreFront. 

Emphasis mine. The Storefront communicates with the Delivery Controller using the XML Service. 

Here’s a list of authentication methods supported by the Storefront. 

When the Storefront communicates the user authentication information to the Delivery Controller, it may or may not include the password too (sent in clear-text) in this communication. If “User name and password” or “Pass-through from NetScaler” is selected, then the password is included. If “Domain pass-through” or “Smart card” is selected, then the password is not. The blog post doesn’t say anything about these, but I think “SAML Authentication” (used for ADFS) will not include the password, while “HTTP Basic” will. 

The StoreFront and Delivery Controller communicates twice (the two times I emphasized above). The first time is when the user authenticates and the StoreFront sends this information to the Delivery Controller to get a list of resources. The second time is when the user makes a selection and this information is passed on to the Delivery Controller so that an appropriate host can be selected. In both instances the password could be sent from the StoreFront to the Delivery Controller.

Brief notes on the Citrix STA

Wanted to point out this PDF from Citrix on the XenApp/ XenDesktop architecture – especially pages 21, 22 which are on how authentication works. During my Citrix course the instructor had talked about it but like an idiot I didn’t take notes and now I can’t find much info on what he was explaining. 

The part which is of interest to me is the STA (Secure Ticket Authority). 

There’s a couple of steps that happens when a user logs in to access a Citrix solution. First: the StoreFront authenticates the user against AD. Or if the user is accessing remotely, the NetScaler gateway authenticates the user and passes on details to the StoreFront. Then the StoreFront passes on this information to the Delivery Controller so the latter can give a list of resources the user has access to. The Delivery Controllers in turn authenticate the user AD. The Delivery Controller then sends a list of resources the user has access to, to the StoreFront, which sends this on to the user’s Citrix Receiver or Browser. This is when the user sees what is available to them, and can select what they want.

When the user selects what they want, this is information is passed on to the StoreFront, which then passes the info to the Delivery Controller – who then finds an appropriate host that can fulfill the requirement and sends this information to the StoreFront. 

The next step is where the STA comes in. 

In case the user is accessing Citrix locally, the StoreFront can create an ICA file with details of the host and send it over to the user’s Citrix Receiver or Browser and the latter can then directly talk to the VDA agent installed on the host (note the StoreFront & Delivery Controller have no more role to play). But what if the user is accessing remotely? We don’t want to send these sensitive details over the public Internet. So, as a workaround, Citrix creates a “ticket” (which is a randomly generated sequence of 32 uppercase alphabetic or numeric characters) and associates the ticket with the details of the host that the Citrix Receiver or Browser need to contact to access the requested resources. This ticket is what is sent over to Citrix Receiver or Browser in the ICA file, using which it can contact the NetScaler gateway and the NetScaler gateway can validate this and initiate a connection with the VDA on the host on behalf of the user. 

So, as we can see the STA only comes into play in case of remote access. The STA is a part of the Citrix XML Service (once again linking to this excellent post!), which is installed as part of the Delivery Controller (so the STA is a part of the Delivery Controller). It is written as an ISAPI extension (called CtxSta.dll) for the IIS WebServer and runs the /Scripts/CtxSta.dll URL. The STA has an ID called the STA_ID, and this along with the TICKET and an STA_VERSION field are what is put into the ICA file. I am not sure whether the STA requires IIS, or it can run standalone (as I blogged previously the Citrix XML Service can run standalone so I would assume the STA can do the same). The Citrix STA FAQ says IIS is required, but that could be outdated.

The Citrix StoreFront is configured with the STA details in the NetScaler Gateway section (remember you only need to use the STA in case of remote users, for which you would have to configure a NetScaler Gateway). 

Similarly the NetScaler itself is configured with the STA details. 

It is important to keep in mind that there are thus TWO places where the STA details are input, and that the details in both places must be the same. The StoreFront uses its configured details to generate a ticket and put it in the ICA file. And the StoreFront uses its configured details to validate that ticket with an STA and identify what resources it should connect to. If the two details are not identical then you will not be able to launch any resources! (I had this problem at work today which is why I decided to refresh my knowledge about STAs and thought of writing this blog post. If the two details are not identical you will get a “Cannot start App:” error because the ticket the client has cannot be validated or used by the NetScaler). 

Just as an aside to myself – the port used to talk to the VDA is 1494 or 2598. This is the case if the Citrix Receiver or Browser contacts the VDA, or if the NetScaler gateway does so on behalf of these. I like to remember port numbers. :o) 

Also – there is nothing that ties a particular STA generated ticket to the device where the request was made from. That is, in theory a remote user could make a request from Computer A, get the ICA file and run it on Computer B – and NetScaler + STA will happily let the user access resources. A ticket only has a 100 seconds validity, so they’d have to do this switch-over quickly though. ;o) Also, a ticket can only be used once. (Also this and more info are from the very informative Citrix STA FAQ by the way). 

Citrix XML Service headaches

Was setting up a Citrix XenDesktop environment in my test environment past few days and the Citrix XML service has been irritating me. There was no grand fix for the issue, but I spent quite a bit of time banging my head over it (and learnt some stuff along the way) so thought I’d make a post to put it all down.

Whenever I’d connect to the Storefront I get the following error:

I only get this error if you use the Receiver app by the way. If I try and connect via HTML5 you get no error at all. (So when in doubt, try with the Receiver always!)

I noticed that the “Citrix Delivery Service” event logs on the server had messages like these:

An SSL connection could not be established: You have not chosen to trust the issuer of the server’s security certificate, my-CA.. This message was reported from the Citrix XML Service at address https://mydeliverycontroller.mydomain/scripts/wpnbr.dll. The specified Citrix XML Service could not be contacted and has been temporarily removed from the list of active services.

I sorted that by changing all my certificates to SHA1. Turns out the default certificate signature algorithm from a Windows CA since 2008R2 is RSASSA-PSS, and Citrix doesn’t support RSASSA-PSS, so switching the CA to use SHA256 or SHA1 by creating a new CA certificate and server certificates is the way to go. In my case since this was a test lab and I didn’t want to encounter any more errors I went with SHA1.

I was mistaken however as I soon got the following error:

An SSL connection could not be established: An unclassified SSL error occurred.. This message was reported from the Citrix XML Service at address https://mydelivercontroller.mydomain/scripts/wpnbr.dll. The specified Citrix XML Service could not be contacted and has been temporarily removed from the list of active services.

This one had me stumped for a long time. I know all my certificates were proper, and they were bound correctly to IIS, so what was this error about? Moreover it didn’t give much details, and there were not many forum or blog post hits either. Everything looked fine – so what the heck?! If I told the Storefront to communicate with the Delivery Controllers over HTTP instead of HTTPS, things worked. So clearly the problem was with HTTPS.

I was able to visit the XML Service URL too with no certificate errors.

Here’s an excellent post on the Citrix XML Service. The important thing to note is that if the IIS role is already present on the server when the Citrix XML Service is being installed, it integrates with IIS; whereas if the IIS role is not present the Citrix XML Service operates in standalone mode. During my install I didn’t have IIS, but since IIS got installed as part of the install I thought Citrix XML Service must be running integrated – but it does not. In my case Citrix XML Service is running standalone.

Anyways, not a good idea to integrate the Citrix XML Service with IIS, so I am going to leave mine standalone. Here’s a Citrix KB article on how to integrate though. Also, for my own info – the registry keys for the Citrix XML Service are under HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\DesktopServer. Apart from the default ones that are present one can also add two DWORD keys XmlServicesEnableNonSsl and XmlServicesEnableSsl to manipulate whether the Citrix XML Service accepts HTTP and HTTPS traffic. By default both keys are not present and have a value of 1, but changing these to 0 will disable HTTP or HTTPS.

Back to HTTPS and the Citrix XML Service. Since it is not integrated in my case, I should follow the SSL instructions for standalone mode. Roughly:

  1. Install the server certificate as usual.
  2. Note the certificate thumbprint.
  3. Find the Citrix Broker Service GUID. Do this via wmic product list (hat tip to this blog post for the latter idea; alternatively the Citrix article shows how to do this via registry).
  4. Use netsh to bind the two.

In my case the command would be something like this:

I didn’t need to do this as my correct certificate was already bound. It was bound to IIS  I guess (the appid wasn’t that of the Citrix Broker Service) so I double checked by removing binding and creating a new one specifically for the Citrix Broker Service. Still no luck!

If I were running Server 2016 there’s some additional steps to follow. But I am running Server 2012 R2.

My setup was such that I had two Delivery Controllers and one of them had the Storefront. It didn’t make a difference which Delivery Controller I chose to add to the Storefront – it never worked. At the same time, switching to HTTP instead of HTTPS always worked. I had no ideas. I posted to the Citrix forums too but only got one reply. Frustrating!

On a whim I installed Storefront on the second Delivery Controller server to see if that works. And it did. The Storefront on that server was able to talk to either Delivery Controllers with no issue. So the issue wasn’t with the Delivery Controllers. For some reason I had always thought the issue was with the Delivery Controllers (I guess because the error message was from the Citrix XML Server/ Citrix Broker Service and that is a part of the Delivery Controller) but now I realized it was to do with the Storefront. And specifically that particular server. I uninstalled and re-installed the Storefront but that didn’t make a difference.

My next suspect was certificates so I compared the trusted root CAs between the broken server and the working server. I found that the broken server had some of my older root CA certificates (remember I had switched my DC/ CA from SHA256 to SHA1) so maybe that was causing an issue? It also had an extra DigiCert certificate. I removed all these and tried again – and voila! it worked!

I am pretty sure I had manually removed all these older DC/ CA certs, so I am not entirely convinced that is the cause. But it sounds plausible and maybe they came back even though I removed.

Update: I hit upon this error again after I stupidly went and renewed my root CA cert (which is my Domain Controller). Stupid, coz I was doing it just for the heck of it (it’s my test lab after all!) but that broke the certs on the Delivery Controllers/ Store Fronts and I began getting these errors again. As a work around I went have deleted the new certs from the local stores of these (Trusted Root CA and also Intermediate CA) . Am sure it will sync in again, so long term I better regenerate my certs or just turn off SSL internally. Most likely the latter as I am lazy. :p