Microsoft Dynamics CRM (Archived)

Connector is Hell (Slow perf and more)

(0) Share

Report

Posted on by Franck C

405

Hi there,

We were and we still are angry about this product. I explain :

We experiments lots of bugs in v2.xxx, slow processing, sometimes too few information to understand an error, accessing map logs is very very slow even if the MSDI Log tables data is empty (and awful if lots of data), odd sync. errors etc..

So after my holidays I decide search for a new version. MBS Dynamics Connector web page is not enough clear, the page looks the same as last year and I did not understand that it offers last version of the product. (No versionning or history..)

So I download last version (3.0.217.1) to hope it could resolve our problems.

It was a DISASTER.

The product did not upgrade the connector but DELETE it (2.3.1306.1) and CLEAN entirely the program folder and the data in database so we lost all our work and of course lost our data sync between Nav (2009 R2) and CRM (2011).

Hopefully we have system file and database backup.

I decide to keep the new version and to recreate all our adaptater and maps, hoping it could be finally better then before.

One integration is done actually and it is WORSE than the version 2 :

- When the server is loading, windows the connector service does not start automatically (I can start it manually but needs about 10 seconds to start)

- The UI open quickly but navigation is still slow (idem for log access)

- The data synchronisation takes a HUGE TIME, LOT MORE THAN BEFORE, eg :

[NAV Contact Card to Contact] has completed. 1473 record(s) have been written. 16 record(s) have been deleted. 0 record(s) have failed. 0 record(s) were filtered. Total runtime was 1980.0083058 seconds.

1.3 seconds per record !!

- During the sync process, huge load of CPU and high network traffic (2 to 10 % of 1 Gbits network)

We use to make records updates from Nav and with 200 000 Contacts we cannot wait next month to get the update in CRM.

So What could it be done to solve these performance problem ?

We have 2 physical servers

- CRM Web : 2008R2 12 cores 6 Go RAM

- CRM SQL : 2008R2 12 cores 22 Go RAM where connector service is installed

The 2 servers are located in our own datacenter.

Any help is welcome

*This post is locked for comments

I have the same question (0)

All responses (20)

Answers (0)

Sort by

Suggested answer

Community Member on at

Like (0)

Report
Copy link

Link copied!
Hello, I am sorry to hear that you have had a poor experience with Connector for Microsoft Dynamics. I will try to address your questions first and then provide you with some ways you can seek out additional support and with some tools that you can use for troubleshooting your issues.

RE: "MBS Dynamics Connector web page is not enough clear, the page looks the same as last year and I did not understand that it offers last version of the product. (No versionning or history..)"

The download pages for Connector in Partner and Customer Source are updated with the latest version number and with the latest download available. The known issues documents are also updated as are the user guides when we release a new version, however you are correct in that we do not update anything else on those pages unless there are new install requirements. The Connector blog does have a version history posting that is kept up-to-date, which can be found here: http://blogs.msdn.com/b/dynamicsconnector/archive/2012/06/19/connector-for-microsoft-dynamics-version-history.aspx

RE:"The product did not upgrade the connector but DELETE it (2.3.1306.1) and CLEAN entirely the program folder and the data in database so we lost all our work and of course lost our data sync between Nav (2009 R2) and CRM (2011)."

This definitely should not have happened. The upgrade should have processed cleanly and we certainly do have live Microsoft Dynamics NAV customers who have successfully upgraded to V3. I do have two questions in regards to the upgrade issues that you encountered:

Were there any errors encountered during the upgrade that were reported in the installer or event log or in the MSI user interface? (It will be incredibly helpful if you contact support to have these error messages if possible).

Did you perform the upgrade on your live, production environment or was this in a test environment as is suggested in the user guide and V3 release blog post?

RE:"When the server is loading, windows the connector service does not start automatically (I can start it manually but needs about 10 seconds to start)"

The service should start automatically and it is set to do so when it is installed. A start time of 10 seconds is pretty normal since the new service has taken over the caching responsibilities from the client.

Can you verify that the service is set to start automatically?

RE:"The UI open quickly but navigation is still slow (idem for log access)"

We made a change to the caching mechanism in the client as well for V3. In previous versions the client would cache everything up front so it could take several minutes to load, now it caches very little up front and you will see faster start times but a bit of a lag the very first time you move to different maps / nodes in the client. This is because when the focus changes, we are caching at that time. After you have moved to a node, the next time the lag should not occur.

RE:"The data synchronisation takes a HUGE TIME, LOT MORE THAN BEFORE"

Do you happen to have the times for initial integration for the previous integration? 1.3 seconds per record is definitely poor given your server and network specifications, however I do not see the specifications listed for the Microsoft Dynamics NAV server. It is also important to note that after the initial integration has been completed, only modified or newly created records are integrated on an on-going basis.

Have you used a network diagnostic tool to determine where the bottleneck might be? I suggest netmon and fiddler as good tools for this type of troubleshooting.

What is the load on the Microsoft Dynamics CRM and NAV servers while the integration is processing?

Have there been any changes / updates made to either Microsoft Dynamics CRM or NAV since your earlier investigations?

Are all of the servers on the same domain? Has ADFS been setup at all?

Were users logged into Microsoft Dynamics CRM or NAV while the initial integration was running?

Connector for Microsoft Dynamics is a fully supported product and I would definitely encourage you to log a support ticket for these issues, which is going to be your best way to get assistance with your setup and configuration. I suspect that when you contact support the questions that I have posed here will again be asked. The Connector for Microsoft Dynamics blog mentioned earlier also has a feature where you can contact the blog author via email which is also an option for you to use to provide feedback on the product.

Again, I am sorry to hear that you have had a bad experience with Connector for Microsoft Dynamics. As with any integration product or project, a good evaluation of the tool is critical to the success of the project. Connector should be evaluated in a test environment to determine that it will satisfy your requirements and we have found that for 80 - 90% of Dynamics customers it works extremely well. But there will always be situations where a different solution might need to be considered to fully meet that project's requirements.

Was this reply helpful? Yes No
Franck C 405 on at

Like (0)

Report
Copy link

Link copied!
Please find my answers with green italic text format :

Setup

Were there any errors encountered during the upgrade that were reported in the installer or event log or in the MSI user interface? (It will be incredibly helpful if you contact support to have these error messages if possible).
Absolutely none.

Did you perform the upgrade on your live, production environment or was this in a test environment as is suggested in the user guide and V3 release blog post?
Yes. Maintaining mirror test environment need too much time and money costs
I used to work with Nav test databases, CRM test Organizations and Connector test Integrations before applying modifications in production items.

Service

Can you verify that the service is set to start automatically?
Yes it does

Slow Sync

Have you used a network diagnostic tool to determine where the bottleneck might be? I suggest netmon and fiddler as good tools for this type of troubleshooting.
No. By the past I installed a version of fiddler but because the complexity of the tool I did not study it.
I will ask our datacenter technical service about these tools.
The amount of data during sync process is understandable. Network link is direct. Filter problem ?

What is the load on the Microsoft Dynamics CRM and NAV servers while the integration is processing?
Nav Server is installed on the CRM SQL Server. It it not our real production Server. In fact we still use an old version of Navision (3.10 Cside) and we use a middleware Nav 2009 R2 to interface with CRM 2011 :
Nav 3.10 « Sync Dev » Nav 2009 R2 « Dynamics Connector » CRM 2011
So the Nav 2009 R2 environnement is only use as a database buffer and require very few resources.
Nav 3.10 environnement is installed other servers.
Before v3 massive sync was less resources consumption.

Have there been any changes / updates made to either Microsoft Dynamics CRM or NAV since your earlier investigations?
No. During july, I was the only one to manage the environment of sync process.

Are all of the servers on the same domain? Has ADFS been setup at all?
Yes, for several years

Were users logged into Microsoft Dynamics CRM or NAV while the initial integration was running?
Yes, it was during the afternoon. We are on summer and our workforce is divided by 2 (currently 80 users. All of them do not interact with CRM via the connector). Note that I only sync Contacts and Customers (2 ways) for changes happened since last Sunday (“Check for data modified after”). Around 1500 of 260000 total records (4 days with 1.3 sec per record. With v2 it was approximatively 8-10 hours)

Notes :

1. I just come to modify the “Check for data modified after” to set up date with 2 hours back. 55 ever syncronized records were concerned and duration is still the same
2. During v3 installation, MSDI database has been upgraded (with data loss) and new tables have been setup (Scheduling*).
   Could it be structural or data conflict with this upgrade ? (some data tables have not been deleted, eg TaskRegister)
3. A global uninstallation with MSDI database deletion and full reinstallation could it be the solution ?
   If yes, time to recreate maps is too long. V3 offer the possibility to create a map from previously saved map. With new installation and so new integration and adaptaters could it be possible ? considering GUIDs ?
(I check that creating new map from file in another integration failed)
   What about “Save integration to file” ? I do not see an option to reload it. If possible, could it work inside a new connector installation ?
4. During configuration of CRM adapters Ieft the checkbox “Skip complete configuration and only generate entity configurations” blank to reimport the solution in CRM. If I uninstall Connector, what about this solution ?

Support : Partner support currently not working… and suggests the use of the community !...

Was this reply helpful? Yes No
Crystal Stegmiller Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

My name is Crystal Stegmiller and I am a Escalation Support Manager on the Microsoft CRM Technical Support team. My team supports the Microsoft CRM Connector. We do have a few offerings that may help in your scenario.

First, if there are any break/fix issues (errors, area of product not working as intended, etc) my team can assist through a standard support case. In regards to your comment "partner support currently not working" do you have any specific cases you can provide me so I can follow up on those?

We also have Advisory services we offer to partners. Advisory services would cover scenarios such as performance tuning, implementation best practices, training, etc. The services my team provides are up to 20 hours. That type of service may help in this scenario, as often performance optimization is specific to each deployment.

Was this reply helpful? Yes No
Franck C 405 on at

Like (0)

Report
Copy link

Link copied!

Thank you for your offer.

However, our situation seems not to be ordinary.

Rather trying to waste time to identify and in the best case correct the problem, I decide to restart installation from scratch.

I will post the result soon.

Was this reply helpful? Yes No
Franck C 405 on at

Like (0)

Report
Copy link

Link copied!

Product has been reinstalled after full uninstall (SQL MSDI DB Deletion, clean up program files directory and Server restart)

I wonder about SQL Collation.

Connector MSDI DB is set with SQL_Latin1_General_CP1_CI_AS

Our SQL Install is set with French_CI_AI (so CRM DBs are set with this collation)

Our NAV 2009 R2 DB is set with French_100_CI_AI value.

Documentation does not say anything about it.

Seems that other users experiment issues when changing collation of MSDI DB

Still not recommended to alter collation for matching our CRM DB ?

Could Performance issues be related with collation sets ?

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report
Copy link

Link copied!

I am not sure how the different collations would effect this, since it would be up to the Microsoft Dynamics CRM and NAV web services to take the proper actions for dealing with anything regarding SQL server. Again, I highly suggest that you open a support case with Crystal so we can track down any issues that there might be with SQL server collations in these products' APIs.

Was this reply helpful? Yes No
Franck C 405 on at

Like (0)

Report
Copy link

Link copied!

I'm not sure either (about collations)

Connector has now been properly reinstalled and maps have been recreated (or reimport thanks to the "Create new map from file" functionnality)

Unfortunatly, performance is still poor.

Considering the product has been set up correctly and is working fine (do the job with no odd errors till now), it seems that the problem is somewhere inside our IT infrastructure.

I am trying to identify bottlenecks using Fiddler 2 but no event is reported. My skills do not include network analysis...

Could someone can give us some tracks to investigate our performance issue ?

Crystal, can you be of any help in this scenario ?

Was this reply helpful? Yes No
Konstantin Levchenko 1,410 on at

Like (0)

Report
Copy link

Link copied!

Hi Franck,

Just an additional thought about slow integration:

From NAV to CRM - CRM plugins, attached to synchronized Entity, can really delay sync process.

From CRM to NAV - all NAV triggers executed and can affect sync too.

Cheers,

Konstantin

Was this reply helpful? Yes No
Franck C 405 on at

Like (0)

Report
Copy link

Link copied!

Of course.

Very few implementations have been done.

Only customization of entities (mainly new fields on Account and Contact Entity, 3 new 1 to N entity from Account)

No dev between version 2 and 3 of the Connector.

Client UI shows the count of the records on which an update is required.

First, the counter take time to appear and then records are processed much slowly than before (1,3 sec. per record)

Does anyone know if connector is trying to get authentification once per batch or is requiring authentification for each record or permanently during the process ?

Note : we have virtualized Active Directory servers (same LAN)

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report
Copy link

Link copied!

In V3, Connector creates a new instance of the source and destination adapters each time it runs a given map. So I think that this is what you are referring to as "batch". When a map is started, the service creates a new instance of the adapters and object providers for that map and as such will re-authenticate at that point, but when it is processing records it does not authenticate for each record.

Was this reply helpful? Yes No