Dynamics CRM Change Tracking feature for data sync framework

Whenever you have custom integration framework for downstream systems, then you have to have a logic to identify the delta data. Mostly full pull is not going to be a feasible solution for various reasons. So teams managed to store the last run time of job & the next pull will fetch the delta data created/modified after that last run datetime stamp. Microsoft has its own way of giving you that delta data using Change Tracking feature.

We are using Data Export Service for replication of Dynamics CRM 365 online database to Azure SQL box. It has its own merits & demerits like the most important filteredviews are not available to mention. Hence the entitlements of data ie. who can see what based on Dynamics CRM security model is not available straight forward which is a huge plus in on-premise SQL data model. But DES is easy to turn-on, very easy to configure, super easy to monitor for any failures & reprocessing failed records. What I like in DES is it’s almost near-real-time sync & native many to many (N:N) entity is also getting synced. Every single CRUD data including Audit, individual Activity types, POA, etc is possible to sync.

Coming back to the original Change tracking discussion, sometimes developers want to handle the deleted data which is tricky. Refer this Stack Overflow question.

Data export service is using this Change tracking platform functionality to sync deleted data, we can also use that. This will avoid a huge data comparison between two systems to identify the deleted data.

The change tracking feature in Dynamics 365 for Customer Engagement Customer Engagement provides a way to keep the data synchronized in a performant way by detecting what data has changed since the data was initially extracted or last synchronized.

 Sample web api request to get the delta data of an entity:

GET [Organization URI]/org1/api/data/v9.0/accounts?$select=name,accountnumber,telephone1,fax HTTP/1.1Prefer: odata.track-changes

Then you will get a response link with delta token:

"@odata.deltaLink": "[Organization URI]/api/data/v9.0/accounts?$select=name,accountnumber,telephone1,fax&$deltatoken=919042%2108%2f22%2f2017%2008%3a10%3a44"

If there is any deleted record, this above URI will give you the deleted record also, you can identify it with the reason tag:

{ "@odata.context":"[Organization URI]/data/v9.0/$metadata#accounts(name,telephone1,fax)/$delta", "@odata.deltaLink":"[Organization URI]/api/data/v9.0/accounts?$select=name,telephone1,fax&$deltatoken=919058%2108%2f22%2f2017%2008%3a21%3a20","value": [ { "@odata.etag":"W/\"915244\"", "name":"Monte Orton", "telephone1":"555000", "fax":"10101", "accountid":"60c4e274-0d87-e711-80e5-00155db19e6d" }, { "@odata.context":"[Organization URI]/api/data/v9.0/$metadata#accounts/$deletedEntity", "id":"2e451703-c686-e711-80e5-00155db19e6d", "reason":"deleted" } ]}

This will be really helpful when you have a custom integration framework. You can find the full working code sample in below link:

Sample: Synchronize data with external systems using change tracking