 © Rakhesh Sasidharan
|
Posted: February 3rd, 2018 | Tags: e-fuse, frame, hpe, powershell, reset, synergy | Category: Infrastructure | § In the HPE BladeSystem c7000 Enclosures one can do something called an eFuse reset to power cycle any the server blades. I have blogged about it previously here.
Now we are on the HPE Synergy 12000 Frames at work and I wanted to do something similar. One of the compute modules (aka server :p) was complaining that the server profile couldn’t be applied due to some errors. The compute module was off and refusing to power on, so it looked like there was nothing we could do short of removing it from the frame and putting back. I felt an eFuse reset would do the trick here – it does the same after all.
I couldn’t find any way of doing this via an SSH into the frame’s OneView (which is the equivalent of the Onboard Administrator in a c7000 Enclosure) but then found this PowerShell library from HPE. Now that is pretty cool! Here’s a wiki page too with all the cmdlets – a good page to bookmark and keep handy. Using this I was able to power cycle the compute module.
1) Install the library following instructions in the first link.
2) Login.
3) Get a list of the modules in the enclosure (not really required but I did anyways to confirm the PowerShell view matches my expectations).
4) Now assign the enclosure object containing the module I want to reset to a variable. We need this for the next step.
In my case the Synergy 12000 Frame (capital “F”) is made up of two frame enclosures. (The frame enclosure is where you have the compute modules and interconnects and frame link modules etc). The module I want to reset is in bay 1 of frame 2. So below I assign the frame 2 object to a variable.
5) Now do the actual eFuse reset.
The -Component parameter can take as argument Device (for compute modules), FLM (for Frame Link Modules), ICM (for InterConnect Modules), and Appliance (for the Synergy Composer or Image Streamer). The -DeviceID parameter is the bay number for the type of component we are trying to reset (so -Component Device -DeviceID 1 is not the same as -Component ICM -DeviceID 1).
An eFuse reset is optional. You could do a simple reset too by skipping the -Efuse switch. The Appliance and ICM components only do eFuse reset though. I am not sure what a regular (non eFuse) reset does.
Posted: December 26th, 2017 | Tags: powershell, profiles | Category: Citrix | § A while ago I had pointed to a blog post I found wherein the author wrote a script to push registry keys to the NTUSER.DAT profile file of a large number of users. I wanted to try something similar in my own environment and while I didn’t go with the script I found I made up something quick and dirty of my own. I know it isn’t as thorough as the one from that blog post (so I’ll link to it again) but it serves my need. :)
So here’s the deal. I have a bunch of profiles located at “\\path\to\profiles\ctxprofiles$“. It has both v4 and v6 profiles. I’d only like to target the v4 profiles, and that too a specific user for testing. This user’s name contains the word “CtxTest” so I match against it. (Post testing I can remove the pipeline and target everyone).
|
|
Get-ChildItem \\path\to\profiles\ctxprofiles$\*.v4 | ?{ $_.Name -match "CtxTest" } | %{ $DatFile = "$($_.FullName)\UPM_Profile\NTUSER.DAT"; Write-Host -ForegroundColor Green "Filename $DatFile" Write-Host "Loading: " -NoNewline REG LOAD HKU\DefaultUser $DatFile; Write-Host "Importing: " -NoNewline REG IMPORT '\\path\to\a reg file.reg' Write-Host "Unloading: " -NoNewline REG UNLOAD HKU\DefaultUser } |
All I do is get the list of folders, and for each of them load the NTUSER.DAT file from the correct location (it’s under a folder called UPM_Profile as I am using Citrix UPM). I just use the REG commands to load the registry hive, import a registry file, and unload the hive. Easy peasy. No error checking etc. so like I said it’s not as great a script as it can be.
Posted: October 9th, 2017 | Tags: powershell, profiles, registry | Category: Asides, Citrix | § I liked this blog post. That’s something I had thought of trying earlier when I was thinking about registry keys and applying them via GPP vs other methods. For now I am applying a huge bunch of my registry keys via the default profile and if there’s any subsequent changes then I’ll push the change out via GPP (for existing users) and also modify the default profile (for new users). But the geeky method the author followed of loading each user’s NTUSER.DAT and modifying the registry key directly is fun and something I had considered. Only catch though is that this has to be done during a period no users are logged in. Coz of this I don’t think I’ll be trying this in my environment but I liked the idea.
Posted: September 19th, 2017 | Tags: GPO, powershell | Category: PowerShell | § Via PowerShell, of course:
|
|
Get-GPO -All | ?{ ([datetime]::today - ($_.ModificationTime)).Days -le 1 } | ft DisplayName,ModificationTime |
And to make a report of the settings in these same GPOs:
|
|
Get-GPO -All | ?{ ([datetime]::today - ($_.ModificationTime)).Days -le 1 } | %{ Get-GPOReport $_.DisplayName -ReportType HTML } > Report.html |
Yeah, I cheated in the end by using > rather than Out-File. Old habits. :)
Not sure why, but doing a search like Get-GPO -All | ?{ ($_.ModificationTime) -eq [datetime]"18 September 2017" } doesn’t yield any results. I think it’s because the timestamps don’t match. I couldn’t think of a way around that.
Posted: May 13th, 2017 | Tags: powershell, windows updates, wmi | Category: PowerShell, Windows | § Was reading about the WannaCrypt attacks. If you have the MS17-010 bulletin patches installed in your estate, you are safe. I wanted to quickly scan our estate to see if the servers are patches with this. Not my job really, but I wanted to do it anyways.
The security bulletin page lists the actual patch numbers for each version of Windows. We only have Server 2008 – 2016 so that’s all I was interested in.
Here’s a list of the Server name, internal version, and the patch they should have.
- Server 2008 | 6.0.6002 | KB4012598
- Server 2008 R2 | 6.1.7600 | KB4012215 or KB4012212
- Server 2012 | 6.2.9200 | KB4012214 or KB4012217
- Server 2012 R2 | 6.3.9600 | KB4012213 or KB4012216
- Server 2016 | 10.0.14393 | KB4013429
One thing to bear in mind is that it’s possible a server doesn’t have the exact patch installed, but is still not at any risk. That is because since October 2016 Windows patches are cumulative. So if you don’t have the particular March 2017 patch installed, but do have the April 2017 one, you are good to go. The numbers above are from March 2017 – so you will have to update them with patch numbers of subsequent months too to be thorough.
Another thing – I had one server in my entire estate where the patch above was actually installed but turned up as a false positive in my script. Not sure why. I know it isn’t a script issue. For some reason that patch wasn’t being returned as part of the “Win32_QuickFixEngineering” output. Am assuming it wasn’t installed that way on this particular server.
Without further ado, here’s the script I wrote:
|
|
Get-ADComputer -SearchBase "OU=Servers,DC=myDomain,DC=com" -Filter * | ?{ Test-Connection $_.Name -Quiet -Count 1 } | %{ $Computer = $_.Name $WinVer = (Get-WmiObject Win32_OperatingSystem -CN $Computer).Version switch ($WinVer) { '6.0.6002' { if ((Get-WmiObject Win32_QuickFixEngineering -CN $Computer -Filter "HotFixID='KB4012598'" | Measure-Object).Count -eq 0) { "$Computer | Server 2008 | Missing KB4012598 "} } '6.1.7600' { if ((Get-WmiObject Win32_QuickFixEngineering -CN $Computer -Filter "HotFixID='KB4012215' or HotFixID='KB4012212'" | Measure-Object).Count -eq 0) { "$Computer | Server 2008 R2 | Missing KB4012212 or KB4012215 "} } '6.2.9200' { if ((Get-WmiObject Win32_QuickFixEngineering -CN $Computer -Filter "HotFixID='KB4012214' or HotFixID='KB4012217'" | Measure-Object).Count -eq 0) { "$Computer | Server 2012 | Missing KB4012214 or KB4012217 "} } '6.3.9600' { if ((Get-WmiObject Win32_QuickFixEngineering -CN $Computer -Filter "HotFixID='KB4012213' or HotFixID='KB4012216'" | Measure-Object).Count -eq 0) { "$Computer | Server 2012 R2 | Missing KB4012213 or KB4012216 "} } '10.0.14393' { if ((Get-WmiObject Win32_QuickFixEngineering -CN $Computer -Filter "HotFixID='KB4013429'" | Measure-Object).Count -eq 0) { "$Computer | Server 2016 | Missing KB4013429 "} } } } |
That’s all. Nothing fancy.
Turns out I was mistaken in my previous post. A few minutes after enabling inheritance, I noticed it was disabled again. So that means the groups must be protected by AD.
I knew of the AdminSDHolder object and how it provides a template set of permissions that are applied to protected accounts (i.e. members of groups that are protected). I also knew that there were some groups that are protected by default. What I didn’t know, however, what that the defaults can be changed.
Initially I did a Compare-Object -ReferenceObject (Get-ADPrincipalGroupMembership User1) -DifferenceObject (Get-ADPrincipalGroupMembership User2) -IncludeEqual to compare the memberships of two random accounts that seemed to be protected. These were accounts with totally different roles & group memberships so the idea was to see if they had any common groups (none!) and failing that to see if the groups they were in had any common ancestors (none again!)
Then I Googled a bit :o) and came across a solution.
Before moving on to that though, as a note to myself:
- The
AdminSDHolder object is at CN=AdminSDHolder,CN=System,DC=domain,DC=com. Find that via ADSI Edit (replace the domain part accordingly).
- Right click the object and its Security tab lists the template permissions that will be applied to members of protected groups. You can make changes here.
- SDProp is a process that runs every 60 minutes on the DC holding the PDC Emulator role. The period can be changed via the registry key
HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Parameters\AdminSDProtectFrequency. (If it doesn’t exist, add it. DWORD).
- SDProp can be run manually if required.
So back to my issue. Turns out if a group as its adminCount attribute set to 1 then it will be protected. So I ran the following against the OU containing my admin account groups:
|
|
Get-ADGroup -SearchBase "OU=Admin Accounts,DC=domain,DC=com" -Filter 'adminCount -eq 1' | ft Name |
Bingo! Most of my admin groups were protected, so most admin accounts were protected. All I have to do now is either un-protect these groups (my preferred solution), or change the template to delegate permissions there.
Update: Simply un-protecting a group does not un-protect all its members (this is by design). The member objects too have their adminCount attribute set to 1, so apart from fun-protecting the groups we must un-protect the members too.
|
|
Get-ADGroup -SearchBase "OU=Admin Accounts,DC=myDomain,DC=com" -Filter 'adminCount -eq 1' | Set-ADGroup -Clear adminCount Get-ADUser -SearchBase "OU=Admin Accounts,DC=myDomain,DC=com" -Filter 'adminCount -eq 1' | Set-ADUser -Clear adminCount |
Update 2: Found this good post with lots more details. How to run the process manually, what are the default protected groups, etc. Read that post in conjunction with this one and you are set!
Update 3: You can unprotect the following default groups via dsHeuristics: 1) Account Operators, 2) Backup Operators, 3) Server Operators, 4) Print Operators. But that still leaves groups such as Administrators (built-in), Domain Admins, Enterprise Admins, Domain Controllers, Schema Admins, Read-Only Domain Controllers, and the user Administrator (built-in). There’s no way to un-protect members of these.
Something I hadn’t realized about adminCount. This attribute does not mean a group/ user will be protected. Instead, what it means is that if a group/ user is protected, and its ACLs have changed and are now reset to default, then the adminCount attribute will be set. So yes, adminCount will let you find groups/ users that are protected; but merely setting adminCount on a group/ user does not protect it. I learnt this the hard way while I was testing my changes. Set adminCount to 1 for a group and saw that nothing was happening.
Also, it is possible that a protected user/ group does not have adminCount set. This is because adminCount is only set if there is a difference in the ACLs between the user/ group and the AdminSDHolder object. If there’s no difference, a protected object will not have the adminCount attribute set. :)
Today I cracked a problem which had troubled us for a while but which I never really sat down and actually tried to troubleshoot. We had an OU with 3rd level admin accounts that no one else had rights to but wanted to delegate certain password related tasks to our Service Desk admins. Basically let them reset password, unlock the account, and enable/ disable.
Here’s some screenshots for the delegation wizard. Password reset is a common task and can be seen in the screenshot itself. Enable/ Disable can be delegated by giving rights to the userAccountControl attribute. Only force password change rights (i.e. no reset password) can be given via the pwdLastSet attribute. And unlock can be given via the lockoutTime attribute.
Problem was that in my case in spite of doing all this the delegated accounts had no rights!
Snooping around a bit I realized that all the admin accounts within the OU had inheritance disabled and so weren’t getting the delegated permissions from the OU (not sure why; and no these weren’t protected group members).
Of course, enabling is easy. But I wanted to see if I could get a list of all the accounts in there with their inheritance status. Time for PowerShell. :)
The Get-ACL cmdlet can list access control lists. It can work with AD objects via the AD: drive. Needs a distinguished name, that’s all. So all you have to do is (Get-ADUser <accountname>).DistinguishedName) – prefix an AD: to this, and pass it to Get-ACL. Something like this:
|
|
Get-ACL AD:$((Get-ADUser rakhesh).DistinguishedName) |
The default result is useless. If you pipe and expand the Access property you will get a list of ACLs.
|
|
Get-ACL AD:$((Get-ADUser rakhesh).DistinguishedName)| select -Expand Access |
The result is a series of entries like these:
|
|
ActiveDirectoryRights : ReadProperty, WriteProperty InheritanceType : All ObjectType : bf967a0a-0de6-11d0-a285-00aa003049e2 InheritedObjectType : bf967aba-0de6-11d0-a285-00aa003049e2 ObjectFlags : ObjectAceTypePresent, InheritedObjectAceTypePresent AccessControlType : Allow IdentityReference : DOMAIN\User IsInherited : True InheritanceFlags : ContainerInherit PropagationFlags : None |
The attribute names referred to by the GUIDs can be found in the AD Technical Specs.
Of interest to us is the AreAccessRulesProtected property. If this is True then inheritance is disabled; if False inheritance is enabled. So it’s straight forward to make a list of accounts and their inheritance status:
|
|
Get-ADUser -SearchBase "OU=Admin Accounts,DC=myDomain,DC=com" -Filter * | ft samAccountName,@{Name="InheritBlocked"; Expr={(Get-Acl AD:$((Get-ADUser $_.samAccountname).DistinguishedName)).AreAccessRulesProtected}} |
So that’s it. Next step would be to enable inheritance on the accounts. I won’t be doing this now (as it’s bed time!) but one can do it manually or script it via the SetAccessRuleProtection method. This method takes two parameters (enable/ disable inheritance; and if disable then should we add/ remove existing ACEs). Only the first parameter is of significance in my case, but I have to pass the second parameter too anyways – SetAccessRuleProtection($False,$True).
Update: Here’s what I rolled out at work today to make the change.
|
|
Get-ADUser -SearchBase "OU=Admin Accounts,DC=myDomain,DC=com" -Filter * | %{ $ACL = Get-ACL AD:$($_.DistinguishedName); $ACL.SetAccessRuleProtection($false,$true); Set-Acl –AclObject $ACL AD:$($_.DistinguishedName); } |
Update 2: Didn’t realize I had many users in the built-in protected groups (these are protected even though their adminCount is 0 – I hadn’t realized that). To unprotect these one must set the dsHeuristics flag. The built-in protected groups are 1) Account Operators, 2) Server Operators, 3) Print Operators, and 4) Backup Operators. See this post on instructions (actually, see the post below for even better instructions).
Update 3: Found this amazing page that goes into a hell of details on this topic. Be sure to read this before modifying dsHeuristics.
Posted: June 29th, 2016 | Tags: group policy, powershell | Category: PowerShell | § Must have seen this recent Windows update that broke GPOs which were missing the Read permission for the “Authenticated Users” group. Solution is to get a list of these GPOs and add the “Authenticated Users” group to them. Here’s a one liner that gets you such a list –
|
|
Get-GPO -All | %{ $GPO = $_.DisplayName; $Perms = (Get-GPPermissions $GPO -All).Trustee.Name; if (!($Perms.Contains("Authenticated Users"))) { "$GPO" } } > GPOs.txt |
This puts it into a file called GPOs.txt in the current directory. Remove/ Modify that last re-direct as needed.
So here’s the situation. We have a bunch of HP rack enclosures. Some blade servers were moved from one rack to another but the person doing the move forgot to note down the new location. I knew the iLO IP address but didn’t know which enclosure it was in. Rather than login to each enclosure OA, expand the device bays and iLO info and find the blade I was interested in, I wrote this batch file that makes use of SSH/ PLINK to quickly find the enclosure the blade was in.
|
|
@echo off FOR %%I IN (R1E1-OA1,R1E2-OA1,R2E1-OA1,R2E2-OA1) DO ( echo Checking %%I echo y | PLINK.EXE admin@%%I -pw <password> "show ebipa" | findstr "<iLO-Address>" ) |
Put this in a batch file in the same folder as PLINK and run it.
Note that this does depend on SSH access being allowed to your enclosures.
Update: An alternative way if you want to use PowerShell for the looping –
|
|
"R1E1-OA1","R1E2-OA1","R2E1-OA1","R2E2-OA1"| %{ $EncName = $_; Write-Host -ForegroundColor Yellow "Checking $EncName"; .\PLINK.EXE -ssh $EncName -l admin -pw <passwd> "show ebipa" | ?{ $_ -match "<iLO Address>" } } |
Posted: May 17th, 2016 | Tags: powershell, remoting, security, winrm | Category: PowerShell | § Just some links I found on PowerShell remoting security –
Had a request to use SolarWinds to highlight servers in a pending reboot status. Here’s what I did.
Sorry, this is currently broken. After implementing this I realized I need to enable PowerShell remoting on all servers for it to work, else the script just returns the result from the SolarWinds server. Will update this post after I fix it at my workplace. If you come across this post before that, all you need to do is enable PowerShell remoting across all your servers and change the script execution to “Remote Host”.
SolarWinds has a built in application monitor called “Windows Update Monitoring”. It does a lot more than what I want so I disabled all the components I am not interested in. (I could have also just created a new application monitor, I know, just was lazy).
The part I am interested in is the PowerShell Monitor component. By default it checks for the reboot required status by checking a registry key: HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired. Here’s the default script –
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
$Error.Clear(); $regpath = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired"; $res = Test-Path $regpath; if ($Error.Count -ne 0) { Write-Host "Message: $($Error[0])"; exit 1; } if ($res) { write-host "Message: Reboot required." write-host "Statistic: 1" exit 0 } else { write-host "Message: Reboot not required." write-host "Statistic: 0" exit 0 } |
Inspired by this blog post which monitors three more registry keys and also queries ConfigMgr, I replaced the default PowerShell script with the following –
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
$res = $false if (Get-ChildItem "HKLM:\Software\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending" -EA Ignore) { $res = $true } if (Get-Item "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired" -EA Ignore) { $res = $true } if (Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager" -Name PendingFileRenameOperations -EA Ignore) { $res = $true } try { $util = [wmiclass]"\\.\root\ccm\clientsdk:CCM_ClientUtilities" $status = $util.DetermineIfRebootPending() if (($status -ne $null) -and $status.RebootPending) { $res = $true } } catch { } if ($res) { write-host "Message: Reboot required." write-host "Statistic: 1" exit 0 } else { write-host "Message: Reboot not required." write-host "Statistic: 0" exit 0 } |
Then I added the application monitor to all my Windows servers. The result is that I can see the following information on every node –
Following this I created alerts to send me an email whenever the status of the above component (“Machine restart status …”) went down for any node. And I also created a SolarWinds report to capture all nodes for which the above component was down.
Then I assigned this to a schedule to run once in a month after our patching window to email me a list of nodes that require reboots.
Posted: May 8th, 2016 | Tags: powershell, solarwinds, winrm | Category: Infrastructure, Windows | § Added the Exchange 2010 Database Availability Group application monitor to couple of our Exchange 2010 servers and got the following error –
Clicking “More” gives the following –
This is because Solarwinds is trying to run a PowerShell script on the remote server and the script is unable to run due to authentication errors. That’s because Solarwinds is trying to connect to the server using its IP address, and so instead of using Kerberos authentication it resorts to Negotiate authentication (which is disabled). The error message too says the same but you can verify it for yourself from the Solarwinds server too. Try the following command –
|
|
New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://<yourexchsvr>/PowerShell -Authentication Negotiate -Credential (get-credential) |
This is what’s happening behind the scenes and as you will see it fails. Now replace “Negotiate” with “Kerberos” and it succeeds –
|
|
New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://<yourexchsvr>/PowerShell -Authentication Kerberos -Credential (get-credential) |
So, how to fix this? Logon to the remote server and launch IIS Manager. It’s under “Administrative Tools” and may not be there by default (my server only had “Internet Information Services (IIS) 6.0 Manager”), in which case add it via Server Manager/ PowerShell –
|
|
> Import-Module ServerManager > Add-WindowsFeature -Name RSAT-Web-Server |
Then open IIS Manager, go to Sites > PowerShell and double click “Authentication”.
Select “Windows Authentication” and click “Enable”.
Now Solarwinds will work.
Posted: May 2nd, 2016 | Tags: dns, powershell, server 2012 | Category: PowerShell | § I had to create multiple A DNS records of the format below –
|
|
10.240.187.21 for svr01 10.240.187.22 for svr02 ... 10.240.187.29 for svr09 |
Just 9 records, I could create them manually, but I thought let’s try and create en-mass using PowerShell. If you are on Windows Server 2012 and above, you have PowerShell cmdlets for DNS.
So I did the following –
|
|
1..9 | %{ $num = $_; Add-DnsServerResourceRecord -ComputerName dnssvr -ZoneName myzone.com -A -CreatePtr -IPv4Address "10.240.187.2$num" -Name "svr0$num" } |
To confirm they are created, the following helps –
|
|
1..9 | %{ $num = $_; Get-DnsServerResourceRecord -ComputerName dnssvr -Name "svr0$num" -ZoneName myzone.com -RRType A } |
Nice!
Posted: April 25th, 2016 | Tags: dhcp, get-service, powershell, services | Category: PowerShell, Windows | § Use PowerShell –
|
|
Get-DomainController | ft Name,@{Name="DHCP Service"; Expression={(Get-Service -ComputerName $_.Name | ?{ $_.Name -eq "DHCPServer" }).Status}} |
Result is a table of DC names and the status of the “DHCP Server” service. If the service isn’t installed (i.e. the feature isn’t enabled) you get a blank.
Posted: August 23rd, 2015 | Tags: exchange 2010, outlook, powershell | Category: Exchange | § As part of troubleshooting something I needed to quickly find what Outlook rules the user had for deleting messages. So I came up with this one-liner.
|
|
Get-InboxRule -Mailbox mev | ?{ ($_.Enabled -eq "True") -and ($_.DeleteMessage -eq "True") } | fl Name,Description |
The result is a list of rule names and a friendly description of what the rule does.
Run this from the EMS of course.
|