You’re offline. This is a read only version of the page.
Hi, we are experiencing an issue with CRM Online where for authentication we are using an on premise ADFS 3.0 that is federated with the Office 365 tenant. SDK calls will intermittently fail with the following stack trace. This is reproduce-able with the CRM SDK Plugin Registration Tool just making a connection with the CRM org, so we have eliminated any custom code in being the issue. The issue is intermittent so it will work fine for a number of hours or days then all of sudden start failing without any changes to the infrastructure or the calling code. Then hours later it will start working again. It can be reproduced on multiple machines but again is intermittent, so it will work on some and not others but then hours later swapped or working everywhere, or no where.
The ADFS setup is load balanced proxies and 2 backend ADFS members. Load balancer and ADFS members have been investigated and configuration validated. This issue is never apparent to users in other applications that use ADFS or any Office 365 services or during login to the CRM GUI. It only seems to appear with the CRM SDK calls, ADFS otherwise operates fine so it has been hard to point the finger at it or the load balancers.
The metadata reference mentioned in the error is clearly accessible and browsable from the requester so we are a little stuck as where to go with the investigation next. The obvious quick work-around has been to stop using ADFS accounts and just use Microsoft Office 365 cloud accounts, but this is not desirable based on the other integrations being performed.
Share
Report[SocketException (0x2746): An existing connection was forcibly closed by the remote host]
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +249
[IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.]
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +905
System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count) +57
System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) +151
System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) +526
System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) +246
System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) +230
System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) +645
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) +9
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) +79
System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) +1467
System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) +84
System.Net.ConnectStream.WriteHeaders(Boolean async) +825
[WebException: The underlying connection was closed: An unexpected error occurred on a send.]
System.Net.HttpWebRequest.GetResponse() +2104
System.Net.HttpWebRequest.GetResponse() +2551
System.ServiceModel.Description.MetadataLocationRetriever.DownloadMetadata(TimeoutHelper timeoutHelper) +333
System.ServiceModel.Description.MetadataRetriever.Retrieve(TimeoutHelper timeoutHelper) +94
[InvalidOperationException: Metadata contains a reference that cannot be resolved: 'https://fs.domain.com/adfs/services/trust/mex'.]
System.ServiceModel.Description.MetadataRetriever.Retrieve(TimeoutHelper timeoutHelper) +432
System.ServiceModel.Description.MetadataExchangeClient.ResolveNext(ResolveCallState resolveCallState) +202
System.ServiceModel.Description.MetadataExchangeClient.GetMetadata(MetadataRetriever retriever) +157
Microsoft.Xrm.Sdk.Client.ServiceMetadataUtility.RetrieveIssuerEndpoints(EndpointAddress issuerMetadataAddress) +160
Microsoft.Xrm.Sdk.Client.CrossRealmIssuerEndpointCollection.get_Item(Uri key) +126
Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.AuthenticateWithADFSForOrgId(AuthenticationCredentials authenticationCredentials, Uri identifier) +142
Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.AuthenticateOnlineFederationInternal(AuthenticationCredentials authenticationCredentials) +656
Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.Authenticate(AuthenticationCredentials authenticationCredentials) +235
Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.Authenticate(ClientCredentials clientCredentials) +150
Microsoft.Xrm.Client.Services.OrganizationService.CreateUserTokenResponse(CrmConnection connection, IServiceConfiguration`1 config) +660
Microsoft.Xrm.Client.Services.OrganizationService.GetUserTokenResponse(CrmConnection connection, IServiceConfiguration`1 config) +288
Microsoft.Xrm.Client.Services.OrganizationService.ToOrganizationServiceProxy(CrmConnection connection) +272
Microsoft.Xrm.Client.Services.OrganizationService.ToOrganizationService(CrmConnection connection) +15
System.Lazy`1.CreateValue() +853
System.Lazy`1.LazyInitValue() +500
Microsoft.Xrm.Client.Services.InnerOrganizationService.UsingService(Func`2 action) +107
Microsoft.Xrm.Client.Services.CachedOrganizationService.InnerExecute(OrganizationRequest request) +105
Adxstudio.Xrm.Services.OrganizationServiceCache.InnerExecute(TRequest query, Func`2 execute, Func`2 selector) +87
Adxstudio.Xrm.Services.<>c__DisplayClass4`3.<LookupAndInsert>b__2(ObjectCache cache) +337
Microsoft.Xrm.Client.Caching.<>c__DisplayClass6`1.<Get>b__5(String key) +24
Microsoft.Xrm.Client.Threading.<>c__DisplayClass5`1.<Get>b__3() +43
Adxstudio.Xrm.Threading.MonitorLockProvider.Lock(String key, Int32 millisecondsTimeout, Action action) +132
Microsoft.Xrm.Client.Threading.LockProvider.Get(String key, Int32 millisecondsTimeout, Func`2 loadFromCache, Func`2 loadFromService) +211
Adxstudio.Xrm.Services.OrganizationServiceCache.LookupAndInsert(TRequest query, Func`2 execute, Func`2 selector, String selectorCacheKey) +875
Adxstudio.Xrm.Services.OrganizationServiceCache.Get(TRequest query, Func`2 execute, Func`2 selector, String selectorCacheKey) +240
Adxstudio.Xrm.Services.OrganizationServiceCache.InnerExecute(TRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +358
Adxstudio.Xrm.Services.OrganizationServiceCache.Execute(OrganizationRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +164
Adxstudio.Xrm.Services.CompositeOrganizationServiceCache.Execute(OrganizationRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +228
Adxstudio.Xrm.Services.ContentMapOrganizationServiceCache.Execute(OrganizationRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +255
Adxstudio.Xrm.Services.CompositeOrganizationServiceCache.Execute(OrganizationRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +228
Adxstudio.Xrm.Services.PortalBusOrganizationServiceCache.Execute(OrganizationRequest request, Func`2 execute, Func`2 selector, String selectorCacheKey) +255
Microsoft.Xrm.Client.Services.CachedOrganizationService.Execute(OrganizationRequest request, Func`2 selector, String selectorCacheKey) +225
Microsoft.Xrm.Client.Services.CachedOrganizationService.RetrieveMultiple(QueryBase query) +119
Adxstudio.Xrm.Services.CrmOnlineOrganizationService.<>n__FabricatedMethod23(QueryBase ) +59
Adxstudio.Xrm.Services.<>c__DisplayClass21.<RetrieveMultiple>b__20() +62
Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction(Func`1 func) +612
Adxstudio.Xrm.Services.CrmOnlineOrganizationService.RetrieveMultiple(QueryBase query) +334
Adxstudio.Xrm.AspNet.Cms.CrmWebsiteStore`2.get_Websites() +757
Adxstudio.Xrm.AspNet.Cms.WebsiteManager`2.get_Websites() +375
Adxstudio.Xrm.AspNet.Cms.<FindAsync>d__11.MoveNext() +1653
[AggregateException: One or more errors occurred.]
System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) +198
System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) +84
Adxstudio.Xrm.AspNet.Cms.WebsiteManager`2.Find(RequestContext request) +114
Site.Startup.ConfigureAuth(IAppBuilder app) in c:\MasterPortal.Identity\App_Start\Startup.Auth.cs:30
Site.Startup.Configuration(IAppBuilder app) in c:\MasterPortal.Identity\Startup.cs:16
[TargetInvocationException: Exception has been thrown by the target of an invocation.]
System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0
System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +56
System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) +712
System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) +35
Owin.Loader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder) +213
Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup) +1037
Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint() +123
System.Threading.LazyInitializer.EnsureInitializedCore(T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory) +241
Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context) +108
System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers) +571
System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context) +304
System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context) +381
System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext) +444
[HttpException (0x80004005): Exception has been thrown by the target of an invocation.]
System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +627
System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +99
System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +739
*This post is locked for comments
I have the same question (0)
All responses (
Answers (
4
