Contact

Subscribe via Email

Subscribe via RSS/JSON

Categories

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

Unlocking the OnePlus One without wiping data (or at least trying to!)

Unboxed my new OnePlus One and immediately installed the usual apps etc. Then I realized that the device bootloader isn’t unlocked! I don’t know why, but I assumed the OnePlus One came unlocked and rooted. Silly me! All I want to do is root the device (an update file for SuperSU can be downloaded from here), but to do that I can’t use the stock recovery as that performs signature checking and so the update will fail). Option 1 is to install a custom recovery, option 2 is to temporarily boot into a custom recovery. Either of these options will then let me root the device. I prefer the second option, but it doesn’t matter really because to install/ temporarily boot a custom recovery the device must be unlocked! And unlocking means my data will be wiped. 

So … here’s what I did to unlock the device and trying not to lose the data. Note that if you are looking for a tutorial sort of post there are better places on the Internet. This is more of a rambling on what I did. I didn’t succeed in what I set out to, so this could be a waste of your time too. :) Also, I have an older post that explains boot loaders, custom recovery, etc. That’s worth a read.  

First I downloaded the Android SDK on my laptop. I didn’t download the ADT, only the SDK tools. Specifically, I downloaded the zip version of the SDK tools so I can put it in my Dropbox for future use. (The zip version can be found under “View All Downloads and Sizes”).

Downloaded the zip file. Extracted the contents to Dropbox. Ran SDK Manager.exe. This had “Android SDk Tools”, “Android SDK Platform-tools”, and “Android SDK Build-tools” selected as well as a bunch of items under the folder called “Android 5.0 (API 21)”. Plus under the “Extras” folder the “Google USB Driver” and “Android Support Library” too were selected. I un-selected the “Android 5.0” folder and “Android SDK Build-tools” as I don’t need these. I need the “Android SDK Platform-tools” as that contains the tools I want. And the USB driver is good to have too. 

selectionsAfter installing these I went to the Dropbox folder where I had extracted everything, went into the android-sdk-windows\platform-tools sub-folder, pressed the SHIFT key and right clicked within the folder, and selected the option to “Open a command window here”.

After this I went to the OnePlus One “Settings” menu > “About phone” > and tapped 7 times on the “Build number”. This enables the “Developer Options” menu under “Settings”. Went in here and enabled “Android debugging” (it’s unchecked in the screenshot below, I checked it).

Screenshot_2014-12-01-17-25-22

 

Now I connected the phone to my laptop. Maybe you’ll get prompted for a driver install in which case you should choose the one under android-sdk-windows\extras\google\usb_driver. If you don’t get prompted for anything then check the Device Manager. If an unknown device called “A001” is present, update its driver with the one in the above location. 

Assuming all goes well typing adb devices in the command prompt should show the OnePlus One:

ADB is pretty cool in that you can do a lot of stuff to your device with it. For example: boot into the boot loader, boot into recovery, take a device backup, sideload apps, and so on. 

The following command will reboot the OnePlus One into bootloader:

Unlike the Nexus devices I am used to, with the OnePlus One the bootloader doesn’t give any info. It has the Cyanogenmod mascot with the words “fastboot mode” beneath it. 

Once in fastboot mode we can ues a tool called fastboot to manage the device. Think of it as ADB but when in the bootloader mode. Assuming all went well before the following command should show the OnePlus One connected:

At this point I can follow either of the options I mentioned above – install a custom recovery or temporarily boot into a custom recovery. In my case this is where I would have issued a command like this to temporarily boot into a recovery image (after downloading a recovery image such as TWRP and specifying the downloaded path below):

If I wanted to flash the custom recovery over the stock recovery I would have typed the following instead:

If you do this without unlocking the device you will get an error along these lines: 

Unlocking is easy. Type the following command (before typing it read the next few paragraphs though):

At this point the device should be unlocked. I expected it to not reboot – because the first boot after unlocking is when the phone wipes all your data–  but mine rebooted! So I quickly powered it off, then powered it on by pressing the Volume UP & Power keys together. Give a minute and the phone will power on and go into the boot loader (fastboot mode). 

I confirmed that fastboot devices shows the phone, so I booted into a temporary recovery as before:

The fastboot screen icon turned white and after a minute TWRP booted. 

I chose the option to “Wipe”, followed by “Advanced Wipe”, and selected “Cache”. This wiped the cache, after which I rebooted the device. (Before rebooting TWRP pointed out that the device isn’t rooted so it can root for me. I chose to go with that). Unfortunately I think I was too late in powering off the phone before because it turns out everything was wiped! :(

Bummer!

Thankfully I was able to restore from the backup – sort off, because that seemed to break off midway – so finally I just recreated the phone. I had kind of expected this outcome as most data wiping as part of unlocking the bootloader is a pretty standard thing. I was hoping to avoid it through the trick above, but I guess it wasn’t my day. :)

(Note to self: If I try this again try flashing recovery and then unlocking. Maybe then the device won’t reboot).

Before I end, I’d like to point out this forum post. It’s from the OnePlus forum and goes into details of unlocking, rooting, etc. It’s a pretty good post, I discovered it while writing this blog post. 

Notes on Windows RE and BCD

Windows RE

Windows RE (Recovery Environment) is a recovery environment that you boot into when your Windows installation is broken. When the Windows boot loader realizes your Windows installation is broken it will automatically boot into Windows RE. (During boot up the Windows boot loader sets a flag indicating the boot process has started. When the OS loads it clears this flag. If the OS doesn’t load and the computer reboots, the boot loader sees the already set flag and knows there’s a problem. A side effect of this is a scenarios where where the OS starts to load but the machine loses power and so the flag isn’t cleared; later when power returns and the machine is turned on the boot loader notices the flag and loads Windows RE as it thinks the OS is broken).

Screenshots

You can manually boot into Windows RE by pressing F8 and selecting “Repair your computer” from the options menu.

winre-1

The Windows RE menu.

winre-2

Apart from continuing the boot process into the installed OS, you can also power off the computer, boot into a USB driver or network connection, or do further troubleshooting. The above screenshot is from a Windows Server 2012 install. Windows 8 has a similar UI, but Windows 7 (and Windows Server 2008 and Windows Vista) have a different UI (but with similar functionality).

Selecting “Troubleshoot” shows the following “Advanced options”:

winre-5

The startup settings can be changed here or a command prompt windows launched for further troubleshooting.

winre-3

It is also possible to re-image the computer from a recovery image. The recovery image can be on a DVD, an external hard drive, or a Recovery Image partition. It is also possible to store your own recovery image to this partition

winre-4

Location of Windows RE

Windows RE itself is based on Windows PE and is stored as a WIM file. This means you can customize Windows RE by adding additional languages, tools, and drivers. You can even add one custom tool to the “Troubleshoot” menu. On BIOS systems the Windows RE WIM file is stored in the (hidden) system partition. On UEFI systems it is stored in the Windows RE tools partition.

The system partition/ Windows RE tools partition has a folder \Recovery\WindowsRE that contains the WIM file winre.wim and a configuration file ReAgent.xml. On the installed system the \Windows\System32\Recovery\ folder has a ReAgent.xml which is a copy of the file in the system tools/ Windows RE tools partition. The former must be present and have correct entries. Also, for BIOS systems, the system partition must be set as active (and it has an MBR ID of 27 which marks it as a system partition).

Notice the “WinreBCD” ID number in the XML file. Its significance will be made clear later (in the section on BCD).

Managing Windows RE

Windows RE can managed using the \Windows\System32\ReAgentC.exe tool. This tool can manage the RE of the currently running OS and for some options even that of an offline OS. More information on ReAgentC.execommand can be found at this TechNet article. Here are some of the things ReAgentC can do:

  • ReAgentC /enable enables Windows RE. ReAgentC /disable disables Windows RE.

    Both these switches work only against the currently running OS – i.e. you cannot make changes to an offline image. You can, however, boot into Windows PE and enable Windows RE for the OS installed on that computer. For this you’ll need the BCD GUID of the OS (get this via bcdedit /enum /v or bcdedit /store R:\Boot\BCD /enum /v where R:\Boot\BCD is the path to the BCD store – this is usually the system partition for BIOS or the EFS partition for UEFI (it doesn’t have a drive letter so you have to mount it manually)). Once you have that, run the command as: ReAgentC /enable /osguid {603c0be6-5c91-11e3-8c88-8f43aa31e915}

    The /enable options requires \Windows\System32\Recovery\ (on the OS partition) to be present and have correct entries.

  • ReAgentC /BootToRE tells the boot loader to boot into Windows RE the next time this computer reboots. This too only works against the currently running OS – you cannot make changes to an offline image.
  • ReAgentC /info gives the status of Windows RE for the currently running OS. Add a switch /target E:\Windows folder to get info for the OS installed on the E: drive (which could a partition on the disk or something you’ve mounted manually).
  • ReAgentc.exe /SetREimage /path R:\Recovery\WindowsRE\ tells the currently running OS that its Windows RE is at the specified path. In the example, R:\Recovery\WindowsRE would be the system partition or Windows RE tools partition that you’ll have mounted manually and this path contains the winrm.wim file. As before add a switch /target E:\Windows folder to set the recovery image for the OS installed on the E: drive.

Operation failed: 3

On my system ReAgentC was working fine until a few days ago but is now giving the following error:

I suspect I must have borked it somehow while making changes for a my previous post on Hyper-V but I can’t find anything to indicate a problem. Assuming I manage to fix it some time, I’ll post about it later.

BCD

I think it’s a good idea to talk about BCD when talking about Windows RE. The BCD is how the boot loader knows where to find Windows RE, and if the BCD entries for Windows RE are messed up it won’t work as expected.

BCD stands for Boot Configuration Data and it’s the Vista and upwards equivalent of boot.ini which we used to have in the XP and prior days.

Boot process difference between Windows XP (and prior) vs Windows Vista (and later)

Windows XP, Windows Server 2003, Windows Server 2000 had three files that were related to the boot process:

  • NTLDR (NT Loader) – which was the boot manager and boot loader, usually installed to the MBR (or to the PBR and chainloaded if you had GRUB and such in the MBR)
  • NTdetect.com – which was responsible for detecting the hardware and passing this info to NTLDR
  • BOOT.INI – a text file which contained the boot configuration (which partitions had which OS, how long to wait before booting, any kernel switches to pass on, etc) and was usually present along with NTLDR

From Windows Vista and up these are replaced with a new set of files:

  • BootMgr (Windows Boot Manager) – which a boot manager that is responsible for showing the boot options to the user and loading the available OSes. Under XP and prior this functionality was provided by NTLDR (which also loaded the OS) but now it’s a separate program of its own. While NTLDR used to read its options from the BOOT.INI file, BootMgr reads its options from the BCD store.
  • BCD (Boot Configuration Data) – a binary file which replaces BOOT.INI and now contains the boot configuration data. This file has the same format as the Windows registry, and in fact once the OS is up and running the BCD is loaded under HKEY_LOCAL_MACHINE\BCD00000000.

    The BCD is a binary file that’s stored in the EFS partition on UEFI systems or in the system partition in BIOS systems under the \Boot folder (it’s a system hidden file so not visible by default). It is a binary file (unlike BOOT.INI which is a text file) so the entries in it can’t be managed via notepad or any text editor. One has to use the BCDEdit.exe tool that’s part of Windows or via third-party tools such as EasyBCD.

  • winload.exe – I mentioned earlier that the boot manager functionality of NTLDR is now taken up by BootMgr. What remains is the boot loader functionality – the task of actually loading the kernel and drivers from disk – and that is now taken care of by winload.exe. In addition, winload.exe also does the hardware detection stuff that was previously done by NTdetect.com.

Vista: the misunderstood Windows

I think this is a good place to mention that while Windows Vista may have been a derided release from a consumer point of view, it was actually a very important release in terms of laying the foundations for future versions of Windows.

Once upon a time we had MS-DOS and Windows 3.x and Windows 95, 98, ME. These had a common set of technologies. Then there was Windows NT, which was different from the these.

Windows 2000 “married” Windows NT and Windows ME. It laid a new foundation upon which later OSes such Windows 2000, Windows XP, and Windows Server 2003 were based. All of these are based on Windows NT and have a common set of technologies. You know one of these, you can work around the others through a bit of trial and error. Some features may be added or missing, but more or less you can figure things out.

Then came Windows Vista and Server 2008. While these are still similar to Windows XP and Windows Server 2003, they are very different too in a lot of ways. Windows Vista and Server 2008 laid the foundations for changes that were further refined in Windows 7, Windows 8, Server 2008 R2, and so on. For instance changes such as WIM files, the boot process, UAC, deployment tools, CBS (Component Based Servicing), and so on. If the only thing you have worked on is Windows XP sure you can get around a bit with Windows Vista or 7, but as you start going deeper into things you’ll realize a lot of things are way different.

Back during the BOOT.INI days you specified disks and partitions in terms of numbers. The BIOS assigned numbers to disks and the BOOT.INI file had entries such as multi(0)disk(0)rdisk(0)partition(1)\WINDOWS which specified the Windows folder on a partition (in this case the 1st partition of the 1st disk) that was to be booted. This was simple and did the trick mostly, except for when you moved disks around or add/ deleted partitions. Then the entry would be out of date and the boot process will fail.

BCD does away with all this.

BCD uses the disk’s GPT identifier or MBR signature to identify the disk (so changing the order of disks won’t affect the boot process any more). Further, each boot entry is an object in the BCD file and these objects have unique GUIDs. (These are the objects I showed through the bcdedit.exe /enum all command above). The object contains the disk signature as well as the partition offset (the sector from where the partition starts on that disk) where it’s supposed to boot from. Thus to boot any entry all BootMgr needs to do is scan the connected disks for the one with the matching signature and then find the partition specified by the offset. This makes BCD independent of the disk numbers assigned by BIOS and it is unaffected by changes made to the order of disks.

A downside of BCD is that while with BOOT.INI one could move the OS to a different disk with the same partitioning and hope for it to boot, that won’t do with BCD as the disk signatures won’t match. BootMgr will scan for the disk signature in the BCD object, not find it, and complain that it cannot find the boot device and/ or winload.exe. (This is not a big deal because BCDEdit can be used to fix the record but it’s something to keep in mind).

Here’s the output from BCDEdit on my machine. There’s two sets of output here – one with a /v switch, the other without.

Couple of things to note here.

First, notice what I meant about each entry being an “object”. As you can see each entry has properties and values – unlike in BOOT.INI days where everything was on a single line with spaces between options.

Second, the /enum switch shows all the active entries in BCD but by default skips the GUID for objects that are universal or known. For instance, the GUID for the boot manager is always {9dea862c-5cdd-4e70-acc1-f32b344d4795} so it replaces that with {default} in the output. Similarly it replaces the GUID for the currently loaded OS – which isn’t universal, but it’s known as it’s the currently loaded one – with {current}. BCDEdit does this to make it easier for the end user to read the output and/ or to refer to these objects when making changes. If you don’t want such “friendly” output use the /v switch like I did in the second case above.

The registry stores the objects as GUIDs. So if I were to take the GUID of the currently running system from the output above and look at the registry I’ll see similar details:

Going back to the BCDEdit output if we compare the device entries for the {bootmgr} and {current} entries we can see it’s represented as partition=\Device\HarddiskVolume1 for the {bootmgr} entry and the friendlier drive letter version partition=C: for the {current} entry (because the partition has a drive letter). BCD starts the volume from 1 so \Device\HarddiskVolume1 refers to the first partition of all the disks on the computer. This is worth emphasising. The \Device\HarddiskVolumeNN representation is not how BCD stores the data internally. Internally BCD uses the disk signature and offset as mentioned earlier, but when displaying to the end-user it uses a friendlier format like \Device\HarddiskVolume1 or a drive letter.

If we compare the registry output above to the corresponding BCD output we can see the partition+disk information represented differently.

Another thing worth noting with the BCDEdit output is that it classifies the output. The first entry is BOOTMRG so it puts it under the section of “Windows Boot Manager”. Subsequent entries are boot loaders so they are put under “Windows Boot Loader”. There’s only one active entry in my system but if I had more entries they too would appear here.

Note that I said there’s only one active entry in my system. There are actually many more entries but these are not active. For instance, there’s an entry to boot into Windows RE but that’s not shown by default. To see all these other entries the /enum switch takes various parameters. For example: /enum osloader shows all OS loading entries, /enum bootmgr shows BOOTMGR, /enum resume shows hibernation resume entries, and so on. To show every entry in the BCD use the switch /enum all (and to see what other options are present do /enum /? to get help).

Notice the Windows RE entry above. And notice that its GUID matches that in the ReAgent.xml file of Windows RE.

On my machine I had one more entry initially:

This is an incorrect entry because the GUID of this entry doesn’t match the Windows RE GUID in the ReAgent.xml file so I deleted it:

Speaking of Windows RE, one of the things we can do from Windows RE (and only from Windows RE!) is repair the MBR, boot sector, and BCD with a tool called Bootrec. To fix only the MBR there’s a tool called bootsect which is available in Windows 8 and above (or Windows PE in case of Windows 7). This tool can replace the MBR with BOOTMGR or NTLDR compatible code and is often useful for fixing unbootable systems.

Another useful tool to be aware of is BCDBoot. This tool is used to create a new BCD store and/ or install the boot loader and related files. I used this tool in a previous posts to install the UEFI bootloader and the BIOS bootloader.

Before I conclude I’d like to link to three posts by Mark Minasi on BCD. They go into similar material as what I did above but I feel are better presented (they talk about the various switches for instance, whereas I just mention them in passing):

Finally, BCDEdit too supports options like you could set in BOOT.INI (for example: use a standard VGA driver, disable/ enable PAE, disable/ enable DEP). You set these options via the bcdedit /set {GUID} ... switch, wherein {GUID} is the ID of the boot entry you want to make the settings on and ... is replaced with the options you want to change. See this MSDN article for more information on the options and how to set them. Common BOOT.INI settings and their new equivalents can be found at this MSDN article.

That’s all for now!