Data Migration (Integration) principles in Microsoft Dynamics 365 Deliveries

Hello,

As COVID-19 epidemic is ongoing and keeping us at home, I would like to share my thoughts and experiences about the position of "Data" during Dynamics 365 deliveries from Legacy System to the new solution.

Information is regularly generated by the daily business activities through multiple channels. It is vital to track the information with necessary applications by converting into data then storing among systems for an organization so we should consider data in legacy systems for most Dynamics 365 deliveries.

Since data might be distributed in the systems, It is challenging to consolidate business information that is coming from legacy systems, in the new delivery. I do not remember any CRM projects that were not asked to skip existing customer details in the Microsoft CRM :)

Data are mostly considered very late, last quarter of the project whole life cycle. I think, It must be taken care of early phase of any deliveries and project teams should discuss while designing the Dynamics 365 deliveries.

Here are the itemized criteria to be considered for Data <--> Dynamics 365 relationships

1. Data Ownership: We should clarify ownership of the data before Dynamics 365 project go live. Data integration goes through accordingly.
   
   For instance;
   If the project is about Dynamics 365 CE delivery, Customer information should be handled by only Dynamics 365 CE in the organization whereas Product related details should be kept by Dynamics 365 Finance&Operations, if it is already used by the client.
2. Data Regulation Concerns: Legacy systems frequently do not have clear and defined data regulation(s) compliance or sometimes out-dated or even not comply at all so we should carefully consider regulations for data before migration and we have to clarify data consent.
    
3. Data Mapping (Transitions): Legacy system data structure mostly does not meet Dynamics 365 data character.
   
   For instance;
   Legacy System may keep only customer Full Name into single field while Dynamics 365 CE keeps First Name - Last Name, two separated fields .
   
   This is the simple case. Sometimes we face complex situation like, we may need to map fields of one entity in legacy system into couple entities in target Dynamics 365.
   
   For instance;
   Legacy system may not split customers into corporate and individual type just keep all customers under Account entity so we may need to map Legacy Account entity into Account and Contact different entities in Dynamics 365 CE according to type of customer.
4. Data Cleansing: Data Cleansing (or Data Scrubbing) is the action of identifying and then removing or amending any data within a legacy system that is:
   
   - Incorrect
   - Incomplete
   - Duplicated
   - Irrelevant/unnecessary (under GDPR)
   
   We need to identify and resolve data quality problems during data migration process from legacy system into Dynamics 365 system.
5. Archive: Do we really need to migrate all data from legacy system to the new Dynamics 365? It is crucial to understand demography of data and should not move unnecessary data to the Dynamics 365 so Archiving data could save us huge amount of data to display Dynamics 365 users. Unnecessary information increases confusion for end users.
6. Data Governance:  make organizational's data assets guarantee generally understandable, correct, complete, trustworthy, secure and discoverable. Data should be tested a lot during data migration.
7. Just in Time Data Served: Kindly inviting you to think fetching records from legacy system when they are needed.
   
   For Instance:
   
   Supposing end client uses Dynamics 365 CE so we can start data movement at the first call of customer to call center.
     

Please avoid using of CRUD operations through SQL Statements for any data migration and integration just follow supported way.