Contact

Subscribe via Email

Subscribe via RSS/JSON

Categories

Recent Posts

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

partedUtil and installing ESXi on a USB disk and using it as a datastore

Recently I wanted to install ESXi 6.5 on a USB disk and also use that disk as a datastore to store VM on. I couldn’t get any VMs to run off the USB disk but I spent some time getting the USB disk presented as a datastore so wanted to post that here.

Installing ESXi 6.5 to a USB is straight-forward.

And this blog post is a good reference on what to do so that a USB disk is visible as a datastore. This blog post is about presenting a USB disk without ESXi installed on it – i.e. you use the USB disk entirely as a datastore. In my case the disk already had partitions on it so I had to make some changes to the instructions in that blog post. This meant a bit of mucking about with partedUtil, which is the ESXi command line way of fiddling with partition tables. (fdisk while present is no longer supported as it doesn’t do GPT).

1. First, connect to the ESXi host via SSH.

2. Shutdown the USB arbitrator service (this is used to present a USB disk to a VM): /etc/init.d/usbarbitrator stop

3. Permanently disable this service too: chkconfig usbarbitrator off

4. Now find the USB disk device from /dev/disks. This can be done via an ls -al. In my case the device was called /dev/disks/t10.SanDisk00Cruzer_Switch0000004C531001441121115514.

So far so good?

To find the partitions on this device use the partedUtil getptbl command. Example output from my case:

The “gpt” indicates this is a GPT partition table. The four numbers after that give the number of cylinders (7625), heads (255), sectors per track (63), as well as the total number of sectors (122508544). Multiplying the cylinders x heads x sectors per head should give a similar figure too (122495625).

An entry such as 9 1843200 7086079 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0 means the following:

  • partition number 9
  • starting at sector 1843200
  • ending at sector 7086079
  • of GUID 7086079 9D27538040AD11DBBF97000C2911D1B8, type vmkDiagnostic (you can get a list of all known GUIDs and type via the partedUtil showGuids command)
  • attribute 0

In my case since the total number of sectors is 122495625 (am taking the product of the CHS figures) and the last partition ends at sector 7086079 I have free space where I can create a new partition. This is what I’d like to expose to the ESX host.

There seems to be gap of 33 sectors between partitions (at least between 8 and 7, and 7 and 6 – I didn’t check them all :)). So my new partition should start at sector 7086112 (7086079 + 33) and end at 122495624 (122495625 -1) (we leave one sector in the end). The VMFS partition GUID is AA31E02A400F11DB9590000C2911D1B8, thus my entry would look something like this: 10 7086112 122495624 AA31E02A400F11DB9590000C2911D1B8 0.

But we can’t do that at the moment as the disk is read-only. If I try making any changes to the disk it will throw an error like this:

From a VMware forum post I learnt that this is because the disk has a coredump partition (the vmkDiagnostic partitions we saw above). We need to disable that first.

5. Disable the coredump partition: esxcli system coredump partition set --enable false

6. Delete the coredump partitions:

7. Output the partition table again:

So what I want to add above is partition 9. An entry such as 9 1843232 122495624 AA31E02A400F11DB9590000C2911D1B8 0.

8. Set the partition table. Take note to include the existing partitions as well as the command replaces everything.

That’s it. Now partition 9 will be created.

All the partitions also have direct entries under /dev/disks. Here’s the entries in my case after the above changes:

Not sure what the “vml” entries are.

9. Next step is to create the datastore.

That’s it! Now ESXi will see a datastore called “USB-Datastore” formatted with VMFS6. :)

Firefox Offline Installers

For my own info –

Good to have these in case you are not connected to the Interwebs and wan’t to install Firefox.

Also, this link on how to set a proxy in Firefox for all users.

[Aside] Always offline mode for cached files

I wasn’t aware of this until a few weeks ago. Starting with Windows 8/ Server 2012 there’s an always offline mode for cached files and folders. That’s useful!

FC with Synergy 3820C 10/20Gb CNA and VMware ESXi

(This post is intentionally brief because I don’t want to sidetrack by talking more on the things I link to. I am trying to clear my browser tabs by making blog posts on what’s open, so I want to focus on just getting stuff posted. :)

At work we are moving HPE Synergy now. We have two Synergy 12000 frames with each frame containing a Virtual Connect SE 40Gb F8 Module for Synergy. The two frames are linked via Synergy 20Gb Interconnect Link Module(s). (Synergy has a master/ satellite module for the Virtual Connect modules so you don’t need a Virtual Connect module per frame (or enclosure as it used to be in the past)). The frames have SY 480 Gen 10 compute modules, running ESXi 6.5, and the mezzanine slot of each compute module has a Synergy 3820C 10/20Gb CNA module. The OS in the compute modules should see up to 4 FlexNIC or FlexHBA adapters per Virtual Connect module.

The FlexHBA adapters are actually FCoE adapters (they provide FCoE and/ or iSCSI actually). By default these FlexHBA adapters are not listed as storage adapters in ESXi so one has to follow the instructions in this link. Basically:

1) Determine the vmnic IDs of the FCoE adapters:

2) Then do a discovery to activate FCoE:

As a reference to my future self, here’s a blog post on how to do this automatically for stateless installs.

Totally unrelated to the above, but something I had found while Googling on this issue: Implementing Multi-Chassis Link Aggregation Groups (MC-LAG) with HPE Synergy Virtual Connect SE 40Gb F8 Module and Arista 7050 Series Switches. A good read.

Also, two good blog posts on Synergy:

[Aside] ESXCLI storage commands

Had to spend some time recently identifying the attached storage devices and adapters to an ESXi box and the above links were handy. Thought I should put them in here as a reference to myself.

VCSA 6.5 – Could not connect to one or more vCenter Server systems?

Had to shutdown VCSA 6.5 in our environment recently (along with every other VM in there actually) and upon restarting it later I couldn’t connect to it. The Web UI came up but was stuck on a message that it was waiting for all services to start (I didn’t take a screenshot so can’t give the exact message here). I was unable to start all the services via the service-control command either.

The /var/log/vmware/vpxd/vpxd.log file pointed me in the right direction. Turns out the issue was name resolution. Even though my DNS providing VM was powered on, it didn’t have network connectivity (since vCenter was down and the DNS VM connects to a vDS? not sure). Workaround was to move it to a standard switch and then I was able to start all the VCSA services.

Later on I came across this KB article. Should have just added an entry for VCSA into its /etc/hosts file.

Using PowerShell to rename folders and change ACLs

Here’s something I had to do at work a few weeks back. I wanted to blog about it since then but never got around to it.

We had copied a bunch of folders from one location to another. Since this was a copy the folders lost their original ACLs. I wanted to do two things – 1) the folder names were in the format “LastName, FirstName” and I wanted to change that to “username” (I had a CSV file with mappings so I could use that to do the renaming). 2) I wanted to change the ACLs so the user had modify rights to the folders.

For the first task here’s what I did:

Note that apart from renaming I also move the folder to a different path (coz I had multiple source locations and wanted to combine them all into one).

For the second task here’s what I did:

Later on I realized the folders still had the BUILTIN\Users entity with full control so I had to remove these too. The code for that was slightly different so here goes:

This is a good article on what I was doing above. And this TechNet article is a useful resource on the various rights that can be assigned.

[Aside] Plug-ins not loading in Adobe Reader XI?

Disable Protected Mode:

Thanks to this forum post.

[Aside] How to convert a manually added AD site connection to an automatically generated one

Cool tip via a Microsoft blog post. If you have a connection object in your AD Sites and Services that was manually created and you now want to switch over to letting KCC generate the connection objects instead of using the manual one, the easiest thing to do is convert the manually created one to an automatic one using ADSI Edit.

1.) Open ADSI Edit and go to the Configuration partition.

2.) Drill down to Sites, the site where the manual connection object is, Servers, the server where the manual connection object is created, NTDS Settings

3.) Right click on the manual connection object and go to properties

4.) Go to the Options attribute and change it from 0 to 1 (if it’s an RODC, then change it from 64 to 65)

5.) Either wait 15 minutes (that’s how often the KCC runs) or run repadmin /kcc to manually kick it off

While on that topic, here’s a blog post to enable change notifications on manually created connections. More values for the options attribute in this spec document.

Also, a link to myself on the TechNet AD Replication topology section of Bridge All Site Links (BASL). Our environment now has a few sites that can’t route to all the other sites so I had to disable BASL today and was reading up on it.

MSI, MST, and disabling auto-healing/ self-repair/ “please wait while Windows configures” messages for certain applications

This issue had be irritated for a week. Finally, thanks to this blog post I think I solved it. Fingers crossed. Bloody Darwin descriptors! :)

So what was the issue? I setup Citrix XenApp for Office 2010, FileSite, and some more applications. The first time I’d launch Outlook 2010 and go to FileSite it would do this initial configuring:

Fair enough – I figure it has to do some initial registry key and folder stuff. I can live with that.This initial configuring is not too harmful either except for a bit of delay to the user. Via GPOs I have pushed out registry keys to set my default library and also tweak some FileSite settings, and this initial configuring does not blow away any of that.

Next I launch Word or Excel and they work happily with no initial configuring. However, sometimes – and this seems to be either when I launch Word or Excel soon after I click Outlook, or when I open one of these via Outlook (like I double click on an attachment), or sometimes just randomly – Word and Excel too do this initial configuring. And they blow up all my GPO pushed out registry settings and that irritates the hell out of me!

Something was causing these two to randomly trigger a repair of FileSite and I had no idea what. I figure it must be some registry key or file that is missing and that’s causing Word or Excel to repair FileSite, but I couldn’t find anything. I spent some time with Citrix UPM doing trial and error, checking to see if I was excluding some important folder, but couldn’t find anything. (I know should have just used SysInternals Procmon to find out what was triggering this I guess, but I felt lazy going that route). Then I decided to try and explore the MSI/ MST file itself and see what I can learn.

I used Orca to view the MSI file and apply the MST transforms to see what was happening. Truth be told, while it was good to understand the structure and what one can do, it didn’t really get me anywhere. From the Application logs on my machine I could see messages like these:

And using Orca I was able to track down these features and components and see what files or registry keys they might be looking for. My hunch was that possibly the file or registry key looked for by these components were missing and that was triggering a self-repair for FileSite. But all those keys and files were present on my machine as far as I could tell, so that was a dead-end.

Here’s an amazing article on features and components and how the self-repair feature is triggered when a file it requires is missing. And here’s another amazing post from StackOverflow that generally goes into the topic of self-repair.

Every time you launch an advertised shortcut (essentially a special shortcut that points to a Windows Installer feature and not directly to a file), Windows Installer will verify the installation by checking the “component key paths” for your product. If a discrepancy is found, a repair is triggered to correct the incomplete installation. The “component key paths” are the “key files” specified for the components inside your MSI – there is one per component. Self-repair can also be initiated by someone instantiating a COM server (or attempting to), someone activating a file via its file extension or MIME registration, and a few other ways.

The impression I got from both articles is that there are entry points to an application and it is possible to tell Windows to do an integrity check when accessing the application via these entry points and if the check fails then do a self-repair. I am not sure if the “Please wait while Windows configures …” window is part of the integrity check or self-repair, but considering it was making changes I am sure it was triggering the self-repair anyways. There are a lot of entry points, from the advertised shortcuts mentioned above (see this article for an example) to activating a repair when doing an action such as opening a document or print.

(As an aside, an MSI/ MST has an ALLUSERS property that determines if it is installed per-machine or per-user or both).

While reading on this topic I also came across this informative article on COM and registry stuff. It isn’t directly relevant to self-repair but has an indirect reference. Here’s some bullet points (note: the actual article has a lot more stuff):

  • COM objects (DLLs? also object classes?) are identified by a GUID. This is called a Class Identifier (ClassID). These are stored under HKEY_CLASSES_ROOT\CLSID. In my machine for instance C:\Windows\system32\oleaut32.dll has a ClassID of {0000002F-0000-0000-C000-000000000046} and can be found under HKEY_CLASSES_ROOT\CLSID\{0000002F-0000-0000-C000-000000000046}.
  • Within the CLSID key in the registry there exists a key called InprocServer32 that points to the actual DLL referenced by the ClassID.
  • Since ClassIDs are GUIDs and hence difficult to remember, we also have Programmatic Identifiers (ProgIDs) that are pointers to these. (It’s like DNS basically. The ProgID is the DNS name; ClassID is the IP address). ProgIDs are stored under HKEY_CLASSES_ROOT.

To take an example from my machine: HKEY_CLASSES_ROOT\Citrix.ICAClient is the ProgID for the Citrix ICA client. Within this HKEY_CLASSES_ROOT\Citrix.ICAClient\CLSID gives me its ClassID which is {238F6F83-B8B4-11CF-8771-00A024541EE3}. I can find this ClassID under HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{238F6F83-B8B4-11CF-8771-00A024541EE3} (notice it can also be under Wow6432Node coz my machine is 64-bit).  In this HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{238F6F83-B8B4-11CF-8771-00A024541EE3}\InprocServer32 points to C:\Program Files (x86)\Citrix\ICA Client\wfica.ocx.

Elsewhere in my machine I also have HKEY_CLASSES_ROOT\MIME\Database\Content Type\application/x-ica which has a CLSID value pointing to the above. So this is how MIME types are associated with the OCX file above (whatever that does).

The above article is the first time I came across the Darwin Descriptor. It was briefly mentioned as something that could be in the InprocServer32; but more on that soon.

Searching more on these topics and self-repair I came across this article. I think I was searching for entry points that trigger self-repair and by now had an idea that self-repair could probably be associated with the registry keys I mentioned above. I’d like to excerpt this FAQ from the same article (emphasis mine):

Q1: Who decides what components are “required”, and what is “intact”?
A1: The author of the installation package. Quite often, he simply follows the wizard in his MSI-authoring product without giving full consideration to what is really required, what is not, what should stay intact and what should not. “Intact” means that the specified file or registry key does exist. The contents of the file or the value of the registry key is allowed to change, so if it is changed, this does not make the installation “not intact”.  But if it is deleted, then the installation becomes “not intact”.

Q2: Why does Windows Installer repair product A if I launched product B?
A2: Very often, product A installs various hooks in the system, and they are used by all applications, including product B. For example, Adobe Reader installs certain extensions in Windows Explorer, which allows full-text searching in PDF files – i.e. you can search in Windows Explorer for all files containing a certain phrase, and Explorer will be able to find that phrase in PDF files – thanks to the extension installed by Adobe Reader. Another function of the extension is generating thumbnails of PDF files for Windows Explorer. This means that whenever you open a folder in Windows Explorer, Adobe Reader’s extension gets activated.  And since Adobe Reader has been installed with resiliency, this means that whenever you open a folder, the system verifies if the Adobe Reader extension is intact. If it’s not, then it will launch the self-repair process.

Q3: But why is it launched every time?
A3: Because it either fails to repair, or because it’s trying to repair a file that the application itself later removes during its regular operation. Such files should not be included in resiliency, but nothing prevents the author from including them; and if the author’s InstallShield or Wise warns them that a certain component has no resiliency information (by highlighting components with no keypath), the author will often blindly comply and create keypaths, even when in fact they are not required.

From this article I came across another article by the same author and here he talks about the Darwin Descriptor. Basically, the Windows installer used to be called Darwin and you can have registry keys that look like garbage but which actually tell it to go and do an integrity check and do self-repair if required. I’d suggest going and reading that article as the author explains it clearly but here’s how it helped me.

Let’s start with one of the Application log errors:

Using Orca  I found the component {86BFF64E-1771-4EF1-991C-C6A99F9690A6}.

By the way, Orca calls it ComponentID, so the Component as far as Orca is concerned is “WSO27AddinsShim.dll.471442DE_0EAA_4BBD_B726_5348144025DB”. Just for kicks, note that it points to a directory called “INSTALLDIR.471442DE_0EAA_4BBD_B726_5348144025DB”.

In the Directory table of the MSI I can find that this points to a “INSTALLDIR”, which in turn is a sub-folder of “WORKSITE”, which in turn is a sub-folder of “INTERWOVEN”, which in-turn is in the Program Files folder. So basically “INSTALLDIR.471442DE_0EAA_4BBD_B726_5348144025DB” points to “Program Files\Interwoven\WorkSite”.

I couldn’t capture it in the above screenshot, but that row also contains a column called KeyPath with the following: “wso27addinsshim.dll.471442DE_0EAA_4BBD_B726_5348144025DB”. Looking at the File table I can find that this points to a file called “WSO27A~1.DLL” (the 8dot3 file name) or “WSO27AddinsShim.dll” (the long name) of version 9.0.6.100 and size 123144 bytes – which I confirmed does exist in the directory path above.

OK, so as of now I have identified the component that is giving an error during integrity check and confirmed that the file it refers to actually exists. So there’s no reason for the integrity check to fail, and anyways if it is failing I can convince myself that it is safe to disable it somehow because the file in question does exist. How do I disable that? For this I went to the Registry table in Orca and found the entries created by the “WSO27AddinsShim.dll.471442DE_0EAA_4BBD_B726_5348144025DB” component. Among the many entries it was creating I found one with the CLSID: CLSID\{37a96034-eb52-4509-96a2-8aee5ea2c093}\InprocServer32. So I went over to HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{37a96034-eb52-4509-96a2-8aee5ea2c093}\InprocServer32 and found a value called InprocServer32 with the following: DC3E4j?Pc8'Y%mIPIlQXFileSite_x86>BzW5J%'5q?se1OG%XntL.

From the article above I knew that I knew that this gibberish was basically telling Windows to do an integrity check and repair of the FileSite_x86 feature (and if you read one of the articles I linked to earlier a feature contains other features and components, so by telling Windows to do an integrity check on the FileSite_x86 feature it was basically triggering the integrity check and repair of a whole other bunch of components – see screenshot below from the FeatureComponents table).

Anyhow, to fix this I went ahead and deleted the InprocServer32 value. I did the same for the CLSID of the other component that was giving an error, and with these two fixes I was effectively able to stop the integrity check and self-repair for FileSite from happening when I launched Outlook, Word, or Excel. It remains to be seen if I encounter new issues because I missed out on adding any files or registry keys expected by FileSite that were previously being added by the self-repair, but we’ll cross that bridge when it comes. :)

Update: Some more links I came across later.

 

[Aside] Finding the source of a domain account lockout

An excellent post. Easy and to the point. Wish I had discovered this much before. The upshot is:

  • Enable debugging on a domain controller: nltest /dbflag:0x2080ffff
  • Disable debugging after a bit: nltest /dbflag:0x0
  • Check the logs at %windir%\debug\netlogon.log to find out where/ what is locking the account.

[Aside] NetScaler and WireShark

FYI to myself: NetScaler + WireShark. Lots of useful WireShark tips and tweaks.

[Aside] Profile Manager NTUSER.DAT editing

I liked this blog post. That’s something I had thought of trying earlier when I was thinking about registry keys and applying them via GPP vs other methods. For now I am applying a huge bunch of my registry keys via the default profile and if there’s any subsequent changes then I’ll push the change out via GPP (for existing users) and also modify the default profile (for new users). But the geeky method the author followed of loading each user’s NTUSER.DAT and modifying the registry key directly is fun and something I had considered. Only catch though is that this has to be done during a period no users are logged in. Coz of this I don’t think I’ll be trying this in my environment but I liked the idea.

Citrix – Word could not create the work file

I came across the problem outlined in this blog in our Citrix environment. It’s only got two test users (one of whom is me) and it only happened once to either of us, so I am not sure if it’s really the same thing or not.

The author’s findings are that it is because the %USERPROFILE%\AppData\Local\Microsoft\Windows\INetCache folder is missing. In my case the folder was present when I took a look, but maybe it got created – I dunno. The odd thing in my case was that I was trying to launch both Outlook and Word together and that’s when Word complained. Once I Word opened after Outlook had launched, it was fine. Also, oddly, this wasn’t the first time I had launched Word either. Previous attempts had worked fine.

What I did for now is add the above path to UPM to be synchronized. Hoping that helps. Else I’ll make a GPP like the author has suggested.

[Aside] Citrix StoreFront customizations & tweaks

As a reference to myself for the future: