Dynamics Community Forum Thread Details

Hi all,

I am in the middle of Dynamics 365 FO on-premises installation. When come to "setup standalone service fabric cluster", I get the error when execute the script below

"> .\CreateServiceFabricCluster.ps1 -ClusterConfigFilePath .\ClusterConfig.json "

Anyone can help ?

Error as below:

Do you accept the license terms for using Microsoft Azure Service Fabric located in the root of your package download? If you do not accept the license terms you may not use the software.
[Y] Yes [N] No [?] Help (default is "N"): Y
Trace folder already exists. Traces will be written to existing trace folder: C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces
Creating Service Fabric Cluster...
If it's taking too long, please check in Task Manager details and see if Fabric.exe for each node is running. If not, please look at: 1. traces in DeploymentTraces directory and 2. traces in FabricLogRoot configured in ClusterConfig.json.
Trace folder already exists. Traces will be written to existing trace folder: C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces
Running Best Practices Analyzer...
Opening TraceWriter SFDeployer, path C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces\SFDeployer-637217674971926882.trace
Diagnostics Store is not a file share. Please consider using a file share accessible from all cluster machines, collecting logs in a local folder is inconvenient.
Best Practices Analyzer completed successfully.
Creating Service Fabric Cluster...
Processing and validating cluster config.
Configuring nodes.
Default installation directory chosen based on system drive of machine '192.168.0.18'.
Copying installer to all machines.
Configuring machine '192.168.0.18'.
Configuring machine '192.168.0.96'.
Configuring machine '192.168.0.17'.
Configuring machine '192.168.0.95'.
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217675996380485.trace
Configuring machine '192.168.0.16'.
Configuring machine '192.168.0.19'.
Configuring machine '192.168.0.20'.
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217675996380485.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676154350783.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676154350783.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676155600450.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676155600450.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676156850126.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676156850126.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676498287463.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676498287463.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676501881165.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217676501881165.trace
Machine 192.168.0.18 configured.
Machine 192.168.0.16 configured.
Machine 192.168.0.96 configured.
Machine 192.168.0.17 configured.
Machine 192.168.0.95 configured.
Machine 192.168.0.20 configured.
Machine 192.168.0.19 configured.
Running Fabric service installation.
Successfully started FabricInstallerSvc on machine 192.168.0.16
Successfully started FabricInstallerSvc on machine 192.168.0.95
Successfully started FabricInstallerSvc on machine 192.168.0.18
Successfully started FabricInstallerSvc on machine 192.168.0.20
Timed out waiting for Installer Service to start for machine 192.168.0.19.
Timed out waiting for Installer Service to start for machine 192.168.0.96.
Timed out waiting for Installer Service to start for machine 192.168.0.17.
Timed out waiting for Installer Service to complete for machine 192.168.0.16. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
Timed out waiting for Installer Service to complete for machine 192.168.0.95. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
Timed out waiting for Installer Service to complete for machine 192.168.0.18. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
Timed out waiting for Installer Service to complete for machine 192.168.0.20. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
CreateCluster Error: System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.17'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.RunFabricServices(MachineHealthContainer machineHealthContainer, FabricPackageType fabricPackageType, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInternal>d__2.MoveNext()
---> (Inner Exception #0) System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.17'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #1) System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.19'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #2) System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.96'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #3) System.ServiceProcess.TimeoutException: Timed out waiting for Installer Service to complete for machine 192.168.0.16. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #4) System.ServiceProcess.TimeoutException: Timed out waiting for Installer Service to complete for machine 192.168.0.95. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #5) System.ServiceProcess.TimeoutException: Timed out waiting for Installer Service to complete for machine 192.168.0.18. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

---> (Inner Exception #6) System.ServiceProcess.TimeoutException: Timed out waiting for Installer Service to complete for machine 192.168.0.20. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

Errors occurred during cluster creation.
Cluster deployment failed. Running best effort rollback.
Trace folder already exists. Traces will be written to existing trace folder: C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces
Closing TraceWriter SFDeployer, path C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces\SFDeployer-637217674971926882.trace
Cleaning up faulted installation.
Opening TraceWriter SFDeployer, path C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces\SFDeployer-637217680367981274.trace
Removing configuration from machine 192.168.0.95
Removing configuration from machine 192.168.0.20
Removing configuration from machine 192.168.0.16
Removing configuration from machine 192.168.0.18
Removing configuration from machine 192.168.0.96
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680376106246.trace
Removing configuration from machine 192.168.0.17
Removing configuration from machine 192.168.0.19
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680376106246.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680928356142.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680928356142.trace
Opening TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680934679908.trace
Closing TraceWriter FabricDeployer, path C:\ProgramData\SF\Log\Traces\FabricDeployer-637217680934679908.trace
System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfiguration(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
RemoveNodeConfiguration did not complete successfully on machine 192.168.0.95. Error: System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfiguration(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName)
Configuration removed from machine 192.168.0.18
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(162,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(163,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(164,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(167,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(168,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(169,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:06:37 PG - AXD365ORCH-03 - Error - Kernel crash upload is configured but failed to get kernel crash dump folder.
6/04/2020 11:07:54 PG - AXD365ORCH-03 - Error - Failed to close ContainerActivator. Error FABRIC_E_INVALID_OPERATION
Error occurred. Event logs from all the nodes are copied to C:\Windows\temp\ServiceFabricEventLogs on machine 192.168.0.18. Please investigate, fix the issue and redeploy your cluster.
Configuration removed from machine 192.168.0.16
6/04/2020 11:04:39 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to start for machine 192.168.0.19.
6/04/2020 11:04:39 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to start for machine 192.168.0.96.
6/04/2020 11:04:40 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to start for machine 192.168.0.17.
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(162,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(163,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(164,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(167,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(168,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:38 PG - AXD365ORCH-01 - Error - tconfig.ini(169,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:05:47 PG - AXD365ORCH-01 - Error - Kernel crash upload is configured but failed to get kernel crash dump folder.
6/04/2020 11:06:52 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to complete for machine 192.168.0.16. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
6/04/2020 11:07:00 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to complete for machine 192.168.0.95. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
6/04/2020 11:07:03 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to complete for machine 192.168.0.18. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
6/04/2020 11:07:15 PG - AXD365ORCH-01 - Error - Timed out waiting for Installer Service to complete for machine 192.168.0.20. Investigation order: FabricInstallerService -> FabricSetup -> FabricDeployer -> Fabric
6/04/2020 11:07:16 PG - AXD365ORCH-01 - Error - CreateCluster Error: System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.17'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.RunFabricServices(MachineHealthContainer machineHealthContainer, FabricPackageType fabricPackageType, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInternal>d__2.MoveNext()
---> (Inner Exception #0) System.InvalidOperationException: Cannot start service FabricInstallerSvc on computer '192.168.0.17'. ---> System.ComponentModel.Win32Exception: The service did not respond to the start or control request in a timely fashion
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Start(String[] args)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(String machineName, ServiceController installerSvc, TimeSpan timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndWaitForInstallerService(String machineName, Nullable`1 timeout)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass20_0.<RunFabricServices>b__0(String m)
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

6/04/2020 11:07:16 PG - AXD365ORCH-01 - Error - Errors occurred during cluster creation.
6/04/2020 11:08:23 PG - AXD365ORCH-01 - Error - System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
6/04/2020 11:08:23 PG - AXD365ORCH-01 - Error - System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfiguration(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
6/04/2020 11:08:27 PG - AXD365ORCH-01 - Error - Failed to close ContainerActivator. Error FABRIC_E_INVALID_OPERATION
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(162,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(163,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(164,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(167,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(168,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:18 PG - AXD365ORCH-03 - Error - tconfig.ini(169,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:06:37 PG - AXD365ORCH-03 - Error - Kernel crash upload is configured but failed to get kernel crash dump folder.
6/04/2020 11:08:37 PG - AXD365ORCH-01 - Error - 6/04/2020 11:07:54 PG - AXD365ORCH-03 - Error - Failed to close ContainerActivator. Error FABRIC_E_INVALID_OPERATION
Error occurred. Event logs from all the nodes are copied to C:\Windows\temp\ServiceFabricEventLogs on machine 192.168.0.16. Please investigate, fix the issue and redeploy your cluster.
Configuration removed from machine 192.168.0.20
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(162,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(163,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(164,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(167,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(168,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:24 PG - AXD365AOS-02 - Error - tconfig.ini(169,1) : Error : Dynamic manifest event tagged with 'd' already exists in the static manifest. Did you mean to use the 's' structured trace option?
6/04/2020 11:07:47 PG - AXD365AOS-02 - Error - Kernel crash upload is configured but failed to get kernel crash dump folder.
6/04/2020 11:08:28 PG - AXD365AOS-02 - Error - Failed to close ContainerActivator. Error FABRIC_E_INVALID_OPERATION
Error occurred. Event logs from all the nodes are copied to C:\Windows\temp\ServiceFabricEventLogs on machine 192.168.0.20. Please investigate, fix the issue and redeploy your cluster.
Configuration removed from machine 192.168.0.17
Collecting event logs failed due to error System.IO.FileNotFoundException: Could not find file '\\192.168.0.17\C$\Windows\temp\ServiceFabricEventLog.txt'.
File name: '\\192.168.0.17\C$\Windows\temp\ServiceFabricEventLog.txt'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite, Boolean checkHost)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName) on machine 192.168.0.17. Please go to each node to view event logs.
Configuration removed from machine 192.168.0.19
Collecting event logs failed due to error System.IO.FileNotFoundException: Could not find file '\\192.168.0.19\C$\Windows\temp\ServiceFabricEventLog.txt'.
File name: '\\192.168.0.19\C$\Windows\temp\ServiceFabricEventLog.txt'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite, Boolean checkHost)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName) on machine 192.168.0.19. Please go to each node to view event logs.
Configuration removed from machine 192.168.0.96
Collecting event logs failed due to error System.IO.FileNotFoundException: Could not find file '\\192.168.0.96\C$\Windows\temp\ServiceFabricEventLog.txt'.
File name: '\\192.168.0.96\C$\Windows\temp\ServiceFabricEventLog.txt'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite, Boolean checkHost)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName) on machine 192.168.0.96. Please go to each node to view event logs.
Rollback failed with System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfiguration(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName)
at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__0()
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<BestEffortRollback>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInternal>d__2.MoveNext()
---> (Inner Exception #0) System.InvalidOperationException: Cannot stop FabricInstallerSvc service on computer '192.168.0.95'. ---> System.ComponentModel.Win32Exception: The service has not been started
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.Stop()
at System.Fabric.FabricDeployer.FabricDeployerServiceController.GetServiceInStoppedState(String machineName, String serviceName, Nullable`1 timeout)
at System.Fabric.FabricDeployer.RemoveNodeConfigOperation.RemoveNodeConfigurationXCopy(String machineName, Boolean deleteLog, String nodeName)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfigurationInner(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at System.Fabric.FabricDeployer.ConfigurationDeployer.RemoveNodeConfiguration(Boolean deleteLog, FabricPackageType fabricPackageType, String machineName)
at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<>c__DisplayClass16_0.<RemoveNodeConfigurationAsync>b__1(String machineName)
at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)<---

Closing TraceWriter SFDeployer, path C:\Microsoft.Azure.ServiceFabric.WindowsServer.7.0.470.9590\DeploymentTraces\SFDeployer-637217680367981274.trace
Create Cluster failed. For more information please look at traces in FabricLogRoot.
One or more errors occurred.
Create Cluster failed. Call with -Verbose for more details

Thanks

Regards

Celine