Application of Dual Write: Ticketing system for Dynamics F&O

Dual write finally gives customers the ability to leverage all the benefits provided by each of the Dynamics 365 offerings by creating a one stop destination for all the customer needs. By providing real-time, tightly coupled, and bi-directional integration between Finance and Operations apps and model driven apps, Microsoft can offer all business solutions in one place. Furthermore, integration of Finance and Operations apps with Common Data Service enables Finance and Operations users to take benefit from the extended Power Platform capabilities. Dual write also follows the low code/no code principle, making it extremely simple to implement.

The IDE MCIO Engineering team decided to leverage the benefits of Dual Write and implemented it to integrate its Compass Finance and Operations application with Dynamics 365 CRM to create a Ticketing system for Finance and Operations users. This was done to continue using Finance and Operations for Compass transactions but leveraging CRM for handling cases and support the users. This Ticketing system platform was built in CRM and connected to Compass Finance and Operations for automatic tracking as 80% of the work comprises of transactions being performed in Compass. The ticket or case creation is achieved by integrating a Canvas app in Compass Finance and Operations. This canvas app can simply be added by clicking on the PowerApps icon on the Finance and Operations UI. Whenever the user wants to raise a ticket, they can then open this app (by clicking on the PowerApps icon), enter the required field details and create a new ticket. They can also view existing tickets in the canvas app itself. This ticket is then created as a new Case record in Dynamics CRM and it is only possible because Dual Write ensures that the required entity data is already available and syncing real-time in CRM. The ticket is added to an unassigned queue from where the Case team members can self-assign the ticket to themselves and take care of it. Even after the cases have been created, Compass Finance and Operations users can continue to perform operations which means that there needs to be real-time tracking of updates to data in CRM as well. It is also important to keep checking that all cases are staying within SLAs as updates are being performed. All of this is automatically handled by Dual Write.

Key takeaways from implementation of Ticketing System

1. Filtering the data to be synced

Oftentimes, it may not be required to sync every record of an entity between the two environments linked through Dual Write. One such scenario that came up during the implementation of Ticketing System was when the ‘MCOI Workflow Tracking Status’ entity in Finance and Operations had to be synced with the condition that only the records having the value in the field DocumentType as ‘PA,’ ‘PCO’ or ‘PO’ were required. Thus, while creating this particular entity mapping, a filter was applied (by clicking the filter icon next to the entity name inside the mapping) as follows:

 This would prevent unnecessary records from being synced that could mess up with business logic. Keep in mind that Dual Write lets you use Open Data Protocol (OData) filter expressions while filtering data from Common Data Service entities whereas Finance and Operations entity data filtering resembles range expressions that are used in query language. For example, Common Data Service filter for the above query would look something like: DocumentType eq ‘Purchase orders’ or DocumentType eq ‘Purchase agreement’ or DocumentType eq ‘PCO list.’ Another interesting scenario came up when there was a requirement to sync ESignEventLogs entity in Finance and Operations with two different entities in the Common Data Service environment where both these entities would have different records based on a field value. To achieve this, two entity mappings were created with the same Finance and Operations entity and the necessary filter was applied in each mapping to only sync the required records.

2. Determining the direction of flow of data and dealing with different datatypes

It may not always be required to have a bidirectional flow (selected by default) of data between Finance and Operations and Common Data Service. In fact, in the Ticketing system example, only a unidirectional flow from Finance and Operations is required to send the necessary case related data to Customer Service Hub that is then used for further actions to support the users in the Ticket or Case that they have created. This can be achieved at a very granular level in Dual Write – it can be determined how data should flow between two applications down to the field level in an entity. This means that each field can be set to have its own direction for the flow of data. Some of the available synchronization directions are Unidirectional field assignment (left to right or right to left), Bidirectional field assignment and each of these field assignments can be applied with Transforms as well. Transformations come into a picture when the datatype of one environment is not the exact same in the other environment. This is a common scenario where it is required to mention how Dual Write should handle such data. For instance, while mapping Finance and Operations enum datatype with an option set in CRM, different Text values of enum values will have to be mapped to the Integer values of the option set (Note that the simple Two Option set in CRM has values True/False and not Yes/No or 1/0). This can be achieved using Value Maps transformation. One such example is given here:

3. Using Virtual fields with Dual Write

There are two types of unmapped fields in Dynamics Finance and Operations – computed and virtual fields. These field values are updated using custom X++ code. They do not exist in the SQL table. It is possible to use virtual fields with Dual Write as well. Virtual fields can be populated by overriding the postLoad() method of an entity. Every time data is exported from an entity in Finance and Operations, the postLoad() method populates the virtual fields as per the custom X++ code. When Dual Write performs the initial sync for an entity mapping, it exports all data from Finance and Operations and that populates the virtual fields’ values in Common Data Service as well. However, it is important to note that after the initial sync when the virtual field value is updated in Finance and Operations, it will not trigger Dual Write and the change will not reflect in Common Data Service. This is because there is no actual write to the SQL database happening in case of virtual fields and Dual Write cannot detect this change. This change will be reflected in Common Data Service only when there is some other change that triggers Dual Write.

4. Using doUpdate() vs update()

X++ developers will be familiar with the two database methods – update() and doUpdate(). The update() method makes all the necessary checks before making the change in the database while doUpdate() bypasses all the required condition checks and directly updates a record. The doUpdate() action is equivalent to directly opening the SQL database in the virtual machine server and making an update in the table without checking for any conditions. Developers can use these methods as needed to update the records. However, in case of Dual Write, it was observed that whenever doUpdate() was used, Dual Write did not get triggered and the change was not reflected in Common Data Service. The exact reason for this behavior is unclear but the key takeaway here is that whenever real time data sync is required, it is better to use update() to update the record.

5. Some initial sync errors

As discussed before, initial sync is used to handle the existing records. There are several limitations with initial sync that are good to know before one sets out to configure Dual Write:

• 500,000 is the maximum number of records that can be imported/exported using initial sync. If there are a greater number of records, try using filters to remove unnecessary records or do not use initial sync and export the records separately as a onetime operation.
• If the initial sync is being performed from Finance and Operations to Common Data Service and the export of records takes longer than 5 minutes, then the initial sync may time-out. This time-out can especially happen if virtual fields are being populated in postLoad method as it is an expensive operation, or the filter queries applied have some error.
• Initial sync cannot be used for a single record that may have failed to sync. It is a full push – it will sync all the existing records (or as per the filter applied).
• If initial sync fails, the entity mapping will not go to Running state. If there are errors with existing data records that cannot be resolved, do not run initial sync, and directly run the entity mapping.
• If initial sync fails with an error message such as ‘Copying pre-existing data completed with Errors. For additional details, go to Initial sync details tab,’ try restarting the DIXF service from Lifecycle Services and then trigger initial sync again. In most cases, this should resolve the error.
• If there are integration keys being used for an entity in Finance and Operations. then ensure that the keys are created in CRM environment also before initial sync, otherwise initial sync will throw ‘Project Validation failed’ error. This error can also occur if there are any mandatory fields which are not being synced. Record creation during initial sync will throw an error in that case.

6. Deploying Dual Write mappings to other environments.

Deploying Dual Write changes to other environments is quite simple and can be achieved through solutioning in Common Data Service. Simply add all the Dual Write entity mappings in a solution in Common Data Service that are also needed in other environments. Export the solution and import it in the desired environment. Also ensure that the Dual Write Core Solution from App Source is already present in that environment. Then go to the new environment in Finance and Operations and configure Dual Write in the same way as done before. Once done, select Apply Solution and choose your custom solution. All the entity mapping changes will be added to the new environment.

Conclusion

To conclude, Dual Write is an excellent way to integrate the completely independent data sources of Dynamics Customer Engagement and Finance and Operations app. This integration unleashes the power of Microsoft Dynamics 365, making it the one-stop destination with all business solution offerings. This article presented an application of Dual Write that was implemented by the IDE MCIO Compass Engineering team to create a support Ticketing system for its business users in Dynamics Finance and Operations. During the implementation of Dual Write for this application, several challenges were faced that resulted in key learnings which were explained.