Contact

Subscribe via Email

Subscribe via RSS/JSON

Categories

Recent Posts

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

Unable to access some performance counters remotely

Maybe it helps someone else. I had an issue today where the disk related performance counters were working locally but not remotely. Well, to start with they weren’t working locally either but I realized they had been disabled via a registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PerfDisk\Performance (value Disable Performance Counters was set to 1 – I deleted it to enable).

For remote access you need the Remote Registry and RPC services running. They were, in my case, and I couldn’t find any other issues either. So I gave the Remote Registry service a restart and now I am able to access the rest of the counters remotely.

Found some Google hits that said restarting the Remote Registry temporarily got all the counters working remotely, but they stopped after a while. Am hoping that’s not my issue (as only some of my counters were not working).

Windows Server 2008 and above – low memory

While troubleshooting something I came across this blog today – Detecting Low Virtual Memory Conditions in Windows 2008 and R2.

Basically, since Windows 2008 there’s an inbuilt low memory detection system called RADAR (Resource Exhaustion Detection and Resolution – cool acronym!) that will log such events.

You can find them in the System logs from source Resource-Exhaustion-Detector. These logs give more details too on what’s using the most resources. Apart from that, there’s also logs under Application & Service Logs > Microsoft > Windows > Resource-Exhaustion-Detector > Operational.

An example message from the System logs looks like this:

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: store.exe (6292) consumed 82729553920 bytes, Microsoft.Exchange.ServiceHost.exe (4224) consumed 784441344 bytes, and w3wp.exe (4828) consumed 754692096 bytes.

Clicking on the details tab and switching to XML view gives more details:

(All this and more info can be found in the link I point to – so please check it out).

I was curious on what these figures meant though. Here’s what I understand from this great blog post by Mark Russinovich.

  • Physical memory – we know.
  • Virtual memory – is physical memory plus the page file on disk.
  • The virtual memory is effectively what the OS can commit to any process. Meaning, guarantee that it can provide. So the system commit limit about is basically the virtual memory. (Well not entirely, as the OS needs some physical memory for itself too).
  • Commit charge – the amount of committed memory across all active processes. This can’t exceed the system commit limit of course.
  • When a process commits a region of virtual memory, the operating system guarantees that it can maintain all the data the process stores in the memory either in physical memory or on disk. Not all memory allocated to a process is of the committed type. Mainly private memory and pagefile-backed are of the committed type. The former can be found via tools like Process Explorer. The latter needs some during around using the handles.exe command with the -l switch.
  • The type of memory allocated to a process depends on the sort of request it makes?

[Aside] IE11 does silently ignores file server locations for PAC file

I had encountered this the hard way some months ago, but today I was Googling on this to share the same with a colleague. Starting with IE 11 you cannot use file server locations (e.g. c:\windows\global.pac or \\mydomain\dfs\global.pac) for the PAC file. You have to use an HTTP or HTTPS location (e.g. http://myserver/global.pac).

It is possible to change a registry key to enable this behavior. This and other nuggets of info can be found in this wonderful MSDN article on web proxy configuration.

  • There’s WinINET and WinHTTP proxy settings. WinINET is the one you set via IE. WinHTTP is the one you set via netsh winhttp I think.
  • Firefox uses the WinINET settings if set to use system proxy settings.
  • Proxy settings are per user, but can be changed via a registry key to be for all users of a machine.
  • Automatically detect settings looks for the wpad.<domainname> entry or uses DHCP to get a proxy script URL.

Importing Registry keys via GPP. Also, item-level targeting.

I wanted to do two things. 1) Import a bunch of registry keys for all users. And 2) Set some key values differently for users depending on their location.

Basically these were registry keys containing the details of all our WorkSite databases and I wanted to enable Auto Login to the local WorkSite DMS of each user. So basically import all the registry keys, and if a user’s location is XYZ (preferably identified via an LDAP query to the l attribute) set the DMS of that site to be AutoLogin.

Found this helpful post on how to export a registry key and convert it into an XML file you can copy paste in Group Policy Preferences. Damn! Nice one.

Next I created a copy of the key in question, set it to a higher number in the order (that happens by default when you do a copy) (we need it to be a higher number so that it overrides the other copy of that same key), and in its properties I enabled item-level targeting based on LDAP. Used a query like this: (&(objectClass=User)(sAMAccountName=%USERNAME%)(l=Dubai))

Some screenshots:

Here are the two keys I mentioned. The second one (with a higher order, 9) is set to a different value. The one with higher order will over-ride the one with lower order.

UPDATE: I was wrong. The lower number has higher precedence, so my screenshot above is wrong. I have hence swapped the order. The order is similar to that of GPOs. Lower number wins.

UPDATE 2: Maybe I wasn’t wrong after all. :) Even with the precedence change this didn’t work. I didn’t troubleshoot more (I suck, I know) – instead I just split these into separate GPOs. One to apply all the default settings, and one with higher precedence but scoped to the group I want to apply the different setting to. That did the trick.

I have set the higher order value to target specific users only though.

In this case, if the query searches for an User class object of matching login name and city (the l attribute) set to either Dubai or Abu Dhabi (the two offices for whom I want this key to be different).

[Aside] Configuring Windows Server 2012 R2/ 2016. Start Menu via GPO

Option 1: https://xenappblog.com/2013/customize-windows-2012-start-screen-using-group-policy/ – export the layout, push it out via GPO to all users. Unfortunately, on Server 2012 R2 this means users can’t modify the Start Menu after this.

Option 2: http://microsoftplatform.blogspot.ae/2012/11/predefining-and-customizing-modern-ui.html – much better. You push out a default “template”, but users can customize it further on.

On a related note: http://virot.eu/push-a-solid-colored-background-to-a-windows-server-2012-or-later/ – for customizing the Start menu colors (these are the colors of the background you get when pressing the Windows button on Server 2012 R2. I find it better to modify the colors first so the two registry keys mentioned in this link are created).

[Aside] Using GPP Item-Level targeting to set environment variables based on AD attributes

The subject says it all. Wanted to do this, found the following. Excellent!

Unrelated, but I was Googling on this and I want to put the links somewhere –

Msiexec /fu {ProductCode} : repairs all user-specific registry settings

Msiexec /fup {ProductCode}: repairs all user-specific registry settings and reinstalls missing files

TIL: Windows 10 goes back to adding a .v5 suffix to profiles

So, back in the Windows 7/ Server 2008 era if you had a roaming profile it was always suffixed with a .v2 extension. So if you username was “rakhesh” and your profile path in AD was “\\someservers\profiles“, the actual path created there would be “\\someservers\profiles\rakhesh.v2“. This is because Windows 7/ Server 2008 had a different profile format to Windows XP/ Server 2003 and prior, so Microsoft decided to tack on this extension so there’s no corruption. Neat idea!

But with Windows 8/ Server 2012 and Windows 8.1/ Server 2012 R2 there was no similar extension. So if you started using roaming profiles with these OSes, and you had a mixed environment, you were in for some trouble. Everything would write to the .v2 profile.

Turns out you can apply a hotfix for Windows 8/ Server 2012 and Windows 8.1/ Server 2012 R2, and then create a new registry value UseProfilePathExtensionVersion (of data 1) under HKLM\System\CurrentControlset\Services\ProfSvc\Parameters and this will cause these two OSes to append a .v3 and .v4 suffix respectively to the roaming profiles. Nice!

With Windows 10/ Server 2016 though, the OS goes back to the old behavior of adding a .v5 extension. So no need for any hotfix or registry key changes. Nice!

As an FYI to myself here’s two alternative approaches to Windows 8/ Server 2012 and Windows 8.1/ Server 2012 R2 profile handling if one didn’t want to go the hotfix + registry key change route: this one’s from Microsoft, and this one’s something I found while Googling. I prefer the latter.

Update: To keep things interesting (via) –

  • Windows 10 build 1511 and older use v5 profiles. (These are builds 1507 and 1511 – aka Threshold 1 and Threshold 2).
  • Windows 10 build 1607 and newer use v6 profiles. (Build 1607 is also known as Redstone 1. The next one, build 1703 is known as Redstone 2, and so on).
  • Windows Server 2016 uses a v6 profile.

[Aside] Domain Controller locator

A while ago I was reading up on the Domain Controller process to confirm some stuff before changes I was making at work. Found a couple of good links, still got them open in my browser but before closing them out I thought I should paste them here as a reference to myself. I know most of the info, but occasionally you forget what you know or need a quick confirmation.

DFSR misconception: the hub server does not mean it is the master

Came across this Microsoft blog post by chance. To quote:

If the topology is set up for hub and spoke, and the spoke were to accidentally delete an item, this should not reflect back to the hub, correct? This should be a one way transfer. What we are seeing is our hub replicates out to the spokes perfectly, but if the spoke deletes an item, the item is then deleted from our hub share. It seems to be acting like a full mesh topology, but it was originally set up at as hub and spoke.

The behavior the customer describes is by design. Because DFS Replication is a multimaster replication engine, any change made on any spoke is replicated back to the hub and to the other spokes. To prevent changes from occurring on spokes, we recommend using shared folder permissions.

I too had always thought a hub-spoke design means the hub is the master server. But now I realize how wrong I was. Basically a hub-spoke or full mesh topology only determines the sync path – it does not denote which server is the master and which servers are slaves. DFSR, like AD, has no master or slave.

In a hub-spoke replication topology, two spoke servers will sync with each other via the hub server – that’s all! Neither server is “inferior” to the master in any way.

Event ID 1046 – DHCP server says it is not authorized even though it is authorized!

This problem ate my head for the past 2 days and wasted a lot of time. For such a simple issue it drove me quite mad.

Built a bunch of DCs for our branch offices. One of them gave trouble with the DHCP server. I authorized it successfully, but the service kept complaining that it wasn’t authorized. Event ID 1046.

The DHCP/BINL service on the local machine, belonging to the Windows Administrative domain mydomain.dom, has determined that it is not authorized to start.  It has stopped servicing clients.  The following are some possible reasons for this: 

This machine is part of a directory service enterprise and is not authorized in the same domain.  (See help on the DHCP Service Management Tool for additional information). 

This machine cannot reach its directory service enterprise and it has encountered another DHCP service on the network belonging to a directory service enterprise on which the local machine is not authorized. 

Some unexpected network error occurred.

Did the obvious ones like reboot server :p and restart service :) and un-authorize and re-authorize the server (no errors either time). Also went ahead and removed the role itself and added back. Nothing helped!

Found a helpful post finally that pointed me in the right direction.

  1. I un-authorized the DHCP server.
  2. Opened up AD Sites and Services. 
  3. Browsed to the Services section (which can be enabled from the View menu if not already visible). 
  4. Browsed to the NetServices section within this. 
  5. On the right pane I had an entry for the IP address for the DHCP server I was trying to authorize. Not an entry by name, but by IP. Dunno why. (All other entries were by name, so I am guessing this is a leftover or a mistake by someone in the past). 
  6. I deleted this entry. 
  7. Waited a while, and then authorized the server. 
  8. No errors now!

Screenshot of the offending entry just for the heck of it (the blacked out part was an IP address):

Alternatively one can open ADSI Edit and go to CN=NetServices,CN=Services,CN=Configuration,DC=myDomain,DC=dom. Then delete the entry (as above) from there. 

What’s odd in my case is that the IP that I deleted was assigned to the DHCP server I wanted to authorize. Am guessing the CNF (short for conflict?) following by the GUID indicates some issue.

Windows CLI – find groups you are a member of

I knew of doing a gpresult /v and finding the group membership. An even better (and faster) way is whoami /groups.

Other useful whoami switches.

[Aside] AD Sites, Subnets, Trusts, etc.

  • How Domain Controllers are Located Across Trusts – this is a delightful article. I don’t know why, but I simply loved the way the author presented the information. Very logically written. Wish I could write blog posts with such clarity.
    • Praise aside, it is a good article on how subnet and site definitions are used to find a Domain Controller closest to you, and especially how it works across forest trusts.
  • Using Catch-All subnets in AD – Wanted to know how catch-all subnets in AD Sites will interact with specific ones. This one explained it. The specific one takes precedence. Which is exactly what you want. :)

PowerShell – Find all AD users with ACL inheritance disabled

Quick one-liner to find all AD user objects with ACL inheritance disabled:

Another one:

 

ADFS errors and WID

Spent a bit of time today tracking down an ADFS/ WID issue. Turned out to be a silly one in the end (silly on my part actually, should have spotted the cause right away!) but it was a good learning exercise in the end. 

The issue was that ADFS refused to launch after a server reboot. The console gave an error that it couldn’t connect to the configuration database. The ADFS service refused to start and the event logs were filled with errors such as these:

The Federation Service configuration could not be loaded correctly from the AD FS configuration database.

Additional Data
Error:
ADMIN0012: OperationFault

There was an error in enabling endpoints of Federation Service. Fix configuration errors using PowerShell cmdlets and restart the Federation Service.

Additional Data
Exception details:
System.ServiceModel.FaultException`1[Microsoft.IdentityServer.Protocols.PolicyStore.OperationFault]: ADMIN0012: OperationFault (Fault Detail is equal to Microsoft.IdentityServer.Protocols.PolicyStore.OperationFault).

A SQL operation in the AD FS configuration database with connection string Data Source=np:\\.\pipe\microsoft##wid\tsql\query;Initial Catalog=AdfsConfiguration;Integrated Security=True failed.

Additional Data

Exception details:
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 – Could not open a connection to SQL Server)

The last one repeated many times. 

I hadn’t installed the ADFS server in our firm so I had no clue how it was setup. Importantly, I didn’t know it used the Windows Internal Database (WID) which you can see in the error messages above. It is possible to have ADFS work with SQL for a larger setup but that wasn’t the case here. Following some blog posts on the Internet (this and this) I downloaded SQL Server Management Studio (SSMS) and tried connecting to the WID at the path given in the error (\\.\pipe\microsoft##wid\tsql\query). That didn’t work for me – it just gave me some errors that the SQL server was unreachable. 

BTW, according to one of the blog posts it is better to launch SSMS as the user who has rights to connect to the WID database (the service account under which your ADFS service runs for instance). That didn’t help in my case (not saying the advice is incorrect, my issue was something else). Found a Microsoft blog post too that confirmed I was connecting to the correct server name – \\.\pipe\microsoft##wid\tsql\query for Windows 2012 and above; \\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query for Windows 2003 & 2008 – but no go. 

That’s when I realized the WID has its own service. I had missed this initially. Trying to start that gave an error that it couldn’t start due to a login failure. This service runs under an account NT SERVICE\MSSQL$MICROSOFT##WID and looks like it didn’t have logon as service rights. It looks like someone had played around with our GPOs (or moved this server to a different OU) and this account had lost its rights. 

The fix is simple – just give this account rights via GPO (or exclude the server from whatever GPO is fiddling with logon as a service rights; or move this server to some other OU). Since the NT SERVICE\MSSQL$MICROSOFT##WID is not a regular account, you can’t add it to GPO from any server (because the account is local and will only exist if WID is installed). So I opened GPMC on my ADFS server and modified the GPO to give this account logon as a service rights. 

[Aside] SPNs

Trying to get people at work to clean up duplicate SPNs, and came across some links while reading about this topic. 

From the official MSDN article: A service principal name (SPN) is a unique identifier of a service instance. SPNs are used by Kerberos authentication to associate a service instance with a service logon account. This allows a client application to request that the service authenticate an account even if the client does not have the account name.

Basically when a client application tries to authenticate with a service instance and the domain controller needs to issues it Kerberos tickets, the domain controller needs to know whose password to use for the service instance – is it that of the server where this instance runs, or any service account responsible for it. This mapping of service -> service account/ computer account is an SPN. It’s of the format service/host:port and is associated with the AD account of the service account or computer account (stored in the servicePrincipalName attribute actually).

That’s all!