Demystifying Dual Write in Data Migration and Integration with Dynamics 365 Apps

Khushhal Garg

1,512

Like ( 0 )

Report

Dual-write is an integrated infrastructure that comes ready to use, facilitating almost instantaneous communication between customer engagement applications and finance and operations applications within the Microsoft Dynamics 365 environment.

Let me elaborate further for you.

Purpose and Functionality

Dual-write enables seamless data flow between different applications, allowing information about customers, products, people, and operations to move beyond application boundaries. This integration empowers all departments within an organization.
Specifically, dual-write establishes tightly coupled, bidirectional integration between finance and operations apps (such as ERP systems) and Dataverse (formerly known as Common Data Service). When data changes occur in either system, dual-write ensures that corresponding writes happen in the other system as well.
The result? An integrated user experience across the apps, benefiting both business users and developers.

Infrastructure Aspect

The dual-write infrastructure offers several key features:

Synchronous and Bidirectional Data Flow: Data flows seamlessly between applications in near-real time.
Synchronization Modes: Dual-write supports online and offline/asynchronous modes, allowing for continuous operation.
Activity and Error Logs: Administrators can view combined logs for monitoring and troubleshooting.
Custom Alerts and Thresholds: Configure alerts and notifications based on specific conditions.
User-Friendly UI: Intuitive interface for filtering and transformations.
Table Dependencies and Relationships: Understand how tables relate to each other.
Extensibility: Works with both standard and custom tables and maps.
Application Lifecycle Management: Reliable setup and management for new customers.

Application Aspect

Dual-write maps concepts between finance and operations apps and customer engagement apps. Here are some supported scenarios:

Integrated Customer Master: Access customer data seamlessly.
Unified Product Mastering: Manage product information consistently.
Vendor Master Integration: Access vendor data.
Prospect-to-Cash: Streamline sales processes.
Procure-to-Pay: Connect procurement and financials.
Activities and Notes: Link customer data and documents.
Project-to-Cash: Handle project-related transactions.
Party Concept: Manage multiple addresses and roles.

Historical Challenges

Traditionally, data migration involved extracting information from a source system, transforming its format, and then transferring it to a target system.
However, this conventional method often caused delays, inconsistencies in data, and operational inefficiencies.

Dual Write plays a crucial role in data migration by bridging the gap between source and target systems effectively.

Top Reasons to Use Dual-Write

Data Integration: Dual-write links Microsoft Dynamics 365 applications, allowing different business apps to work together.
Robust Framework: Provides near-real-time, bidirectional integration, reduction in manual entry, errors and expediting processes.
Empowers Business Processes: Ensures consistent data across systems.
Unified User Experience: Users gain from an integrated experience across customer engagement and finance/operations apps, enhancing insights to empower them with informed decision-making.

Dual-write for developers and architects

Purpose and Integration
- Dual-write streamlines data exchange between finance and operations applications and customer engagement platforms by integrating two AppSource solutions within Dataverse.
- These enhancements accommodate enterprise resource planning (ERP) scale by enhancing the table schema, plugins, and workflows within Dataverse
Schema Adjustments
- To align with finance and operations systems, Dual-write introduces significant adjustments to the Dataverse schema. Understanding these modifications can help avoid future redesign and redevelopment efforts.
- New entities like “company” and “party” facilitate seamless interaction between applications built on Dataverse and finance and operations applications.
Data Integrity and Currency Precision
- Users can extend the decimal precision of currency data types in customer engagement applications.
- This ensures data integrity during currency transmission between finance and operations applications and Dataverse.
- Organizations can choose to opt in for up to 10 decimal places of precision, while 4 decimal places remain the default.
Additional Features
- Activities and notes are consolidated and enhanced to cater to both system users (C1s) and system customers (C2s).
- Dataverse incorporates date effectivity, supporting historical, current, and future data within the same tables.
- Product unit conversions are enabled for products, quotes, orders, and invoices.

Regional Support

Currently, dual-write is supported in the following regions:

Asia
Australia
Brazil
Canada
Europe
France
Germany
India
Japan
South America
Switzerland
United Arab Emirates
United Kingdom
United States

System Requirements

Finance and Operations apps that have build version 10.0.9 (10.0.383.20013) (Quality update) and platform update 33 or later
Customer engagement apps that have platform version 9.1.0000.11732 or later

Limitations

Coexistence with Prospect to Cash Solution for Data Integrator
- Dual-write cannot be run simultaneously with the Prospect to Cash solution for Data Integrator. If you are using the latter, you must uninstall it before enabling Dual-write.
Trial Instances of Finance and Operations Apps
- Dual-write setup is not supported on trial instances of finance and operations apps. It requires a non-trial environment.
Single Instance Integration
- Dual-write is designed to integrate a single finance and operations app instance with a single customer engagement app instance. It does not support multiple instances.
Legal Entity Synchronization Limits
- Initial synchronization is limited to 40 legal entities.
- Live synchronization has a maximum limit of 250 legal entities.
Cross-Company Data Sharing
- Dual-write does not support cross-company data sharing. It operates within the boundaries of individual legal entities.
Tenant and Datacenter Requirements
- The finance and operations app and the customer engagement app must be in the same Microsoft Netra tenant.
- Both apps must be deployed in the same Microsoft Azure datacenter.
Event Triggering
- Dual-write is not triggered by doInsert, doUpdate, and doDelete events in finance and operations apps. Instead, use Insert, Update, and Delete events to initiate dual-write synchronization.

Remember that these limitations are important to consider when implementing Dual-write in your environment.

Setup Guidance

The following setup scenarios are supported:

A new finance and operations app instance and a new customer engagement app instance
A new finance and operations app instance and an existing customer engagement app instance
A new finance and operations app instance that has data and a new customer engagement app instance.
A new finance and operations app instance that has data and an existing customer engagement app instance.
An existing finance and operations app instance and a new customer engagement app instance
An existing finance and operations app instance and an existing customer engagement app instance

For details, see Guidance for dual-write setup - Finance & Operations | Dynamics 365 | Microsoft Learn

For Security roles and permissions, see Set up dual-write security roles and permissions - Finance & Operations | Dynamics 365 | Microsoft Learn

Enable table maps for dual-write

Once you activate a table map for dual-write, it initiates in the Not running state. Subsequently, the table map undergoes an initialization phase, during which it performs an initial write operation by duplicating existing data from tables on both ends. Once the table map is fully enabled, it transitions its status to Running.

Status	Description	Available actions
Not running	The table hasn't yet been enabled for dual-write. Every table begins at the Not running status.	Run
Initializing	The initial write is occurring.	None
Running	The table is enabled for dual-write.	Stop, Pause
Paused	The table is in a paused state, and all new requests are queued.	Run
Resuming	The table is catching up on rows that were queued while the table was paused.	None

Samples: Enabling the Customers V3—Contacts table map

Customization Support

Dual-write offers preconfigured maps for certain business processes. Nonetheless, there may arise situations necessitating additional fields, maps, or transformations. The dual-write platform is designed to be extensible, allowing users to create custom maps and enhance existing ones with custom fields to synchronize data between finance and operations applications and Microsoft Dataverse.

For details, see Customization guidance for dual-write - Finance & Operations | Dynamics 365 | Microsoft Learn

Customize table and column mappings - Finance & Operations | Dynamics 365 | Microsoft Learn

Initial Synchronization Approach

The initial synchronization facilitates the bidirectional copying of existing data between applications, with various considerations to bear in mind during execution. For instance, it may be necessary to migrate data prior to the go-live phase. In such instances, data can be transferred into one side through data migration, subsequently synchronized to the other side through the initial synchronization process.

Following approaches are recommended for the initial synchronization.

Single-threaded tables: Begin by migrating data into the finance and operations application, followed by initiating the initial synchronization process to transfer the data to Dataverse. According to Microsoft's lab testing, this sequence demonstrates better performance compared to synchronization from Dataverse to finance and operations applications.

Multi-threaded tables: Start by transferring data into Dataverse, then initiate the initial synchronization to migrate the data to the finance and operations application.

Constraints

Data migration slow-down with enabled dual-write
500,000 rows per run limitation
24-hour limit
40 legal entities while the environments are being linked limitation
Limitation to support table maps that have 10 or more lookups
Five-minute limit for finance and operations data export
Security role for write access
Company and Currency Exchange Tables Required Security Role
Full push only for Initial sync
Top 5 error viewing limitation
Double quotes limitation
Live Synchronization limits

For details, see Initial Synchronization Constraints and Live Synchronization Limits

Challenges/ Risks

Performance: Large volumes of data may result in synchronization lags or delays.
Customizations: Complex customizations in either Dynamics 365 F&O or Dynamics 365 CE could potentially interfere with the seamless operation of Dual Writes.
Data Update Conflicts: Without proper handling of simultaneous data edits could result in data inconsistency.
Data Integrity: Data corruption is possible in the absence of accurate field mappings and validations.

With a proper strategy for monitoring, reviewing customizations, accurate field mapping, and handling conflicts, the risks associated can be minimized.

In conclusion, Dual-write simplifies data migration and ensures that your organization’s various departments have access to accurate and up-to-date information across different Dynamics 365 apps.