The Microsoft.Crm.Setup.Shared.RemoveOrphanedProductUpdateRegistryKeysAction occurs during the rollback of the install, so if you're at this point you've already failed to install and now an error is occurring during the uninstall.
The error during install can be caused by various things - some posters on this thread have mentioned that for them they were missing or required a newer version of sqlncli10.dll
For me, the error was found in the logs here, notice that it stops executing Install actions and starts executing Uninstall actions
23:01:49| Info| CrmAction execution time; MsiInstallServerAction; 00:00:40.1565203
23:01:49| Info| Executing Install action: Microsoft.Crm.Setup.Server.LoadPlatformLibrariesAction
23:01:49| Info| CrmAction execution time; LoadPlatformLibrariesAction; 00:00:00.0937502
23:01:49| Error| Installer Complete: ServerMsiInstaller - Error encountered
23:01:49|Warning| Error reported while configuring _Files. Attempting rollback
23:01:49| Info| ServerMsiInstaller: Beginning uninstall operation
23:01:49| Info| Executing Uninstall action: Microsoft.Crm.Setup.Server.UnregisterRoleAction
Decompiling Microsoft.Crm.Setup.Server shows that this calls RegControl.LoadLibrariesServer() which is in Microsoft.Crm.Core
Decompiling Microsoft.Crm.Core shows that it tries to load
"Bin\CRMErrors.dll",
"Bin\CRMCore.dll",
"Bin\CrmOfflineSync.dll",
"Bin\CRMWatson.dll",
"Bin\sqmapi.dll",
"Bin\osafehtm.dll"
From the install directory as found in registry key "HKLM\SOFTWARE\Microsoft\MSCRM" .... by default "C:\Program Files\Microsoft Dynamics CRM"
From there, it's over to PROCMON.exe to find out where it's failing.
PROCMON shows that when loaded, both CRMCore.dll and CRMWatson.dll are failing to load MSVCP120.dll
