Have you ever asked yourself the question – “Is Microsoft Flow suitable for integrating Dynamics 365 with other systems?”.

There are scenarios where Microsoft Flow is suitable for Dynamics 365 integrations. There are also many scenarios where it isn’t. Flow is a very powerful integration tool and can provide huge benefit with little effort and cost. However, there are a lot of things to consider when connecting Flow to Dynamics 365 and this article aims to tackle them.

The discussion points for Dynamics 365 integration with Microsoft Flow are:

First, let’s start with why would we even consider Microsoft Flow as an integration tool for Dynamics 365, isn’t it primarily a workflow engine?

Note: The term Dynamics 365 in this article can also be referred to as Model-Driven PowerApps since they are essentially are one in the same. They both use the Common Data Service (CDS) as their underlying database layer and CDS is the thing that Microsoft Flow ultimately connects to.

Typical Dynamics 365 Integration Obstacles

Based on my Dynamics 365 consulting experience, custom code or middleware integrates other systems with Dynamics 365, in most instances. For example, Scribe, BizTalk or SQL Server Integration Services (SSIS), Azure LogicApps etc. These integrations are typically expensive to implement and maintain. They also require specialist skills which can be hard to come across and expensive.

From my experience, a large proportion of the costs of any Dynamics 365 (CRM) implementation project and the ongoing maintenance is due to integration.

What if integrations could be simpler and more cost effective? Wouldn’t it be great if Microsoft’s customers could implement and manage their own integrations? And expensive specialist skills not required?

When to use Microsoft Flow for Dynamics 365 Integration?

Let’s face it, Microsoft Flow is not an enterprise integration tool by any means, but there are some use cases I think Flow is appropriate for integration. Here are some examples:

  • For Proof of Concepts (POCs), pilots etc. to test out an integration scenario.
  • To quickly and cost effectively add valuable information to Dynamics 365 from other systems. In CRM terms, giving the full 360 degree view of the customer in one place.
  • When the source and destination systems have Microsoft Flow connectors available. There are over 200 connectors! However, if you have a technical skills then it is not difficult to create a custom connector.
  • For old school flat file integration. If connectors are not available and you do not have the ability to create a custom connector then you can use Microsoft Flow’s connectors and actions for parsing text files.

Guidelines

Here are some guidelines that I use when considering Microsoft Flow for integration based on the integration requirements.


Integration Requirement

Explanation
AsynchronousMicrosoft Flow’s triggers and processing are asynchronous in nature. Flow cannot be used for synchronous (real time) integrations.
Not Business CriticalMicrosoft Flow is not an enterprise integration tool so I would not use it for business critical scenarios.
e.g. financial type integrations which are critical to running the business.
Small volumes of dataMicrosoft Flow is relatively slow compared to other integration technologies and can take a while to process large volumes of data.
Simple integration logic \ data transformationMicrosoft Flow can handle different types of logic and data transformation through expressions and condition actions. However, the Flow configuration can get complex and difficult to maintain.
Low transaction volumeMicrosoft Flow has limitations on how much data it can process and for how long. There are also limitations on throughput and how often Flows can be triggered which are tied to the type of Flow plan that a user has.
Simple Application Lifecycle Management (ALM)The ALM process for Microsoft Flow is currently manual. Flows are created in a Sandbox environment and then exported\imported into other environments. Microsoft Flows can now be added to Dynamics 365 \ PowerApps Solutions.
Simple Dynamics 365 configurationCare needs to be taken when Flow inputs data into Dynamics 365 that it does not inadvertently trigger workflows and plugins that should not be. This is discussed further in the Considerations section.

Flow Connectors for Dynamics 365

Microsoft Flow uses connectors to talk to over 200 different systems. For Dynamics 365 and the Common Data Service (CDS), the database layer underlying Dynamics 365, there are a number of different connectors that can be used. The Dynamics 365 connector is deprecated, as stated in the Microsoft Flow product update article for April. The Common Data Service connector should be used by Flow for interacting with Dynamics 365.

Flow Integration Triggers

Sending data from one system to the other requires a trigger i.e. a trigger to tell Flow when to run the integration. Microsoft Flow has a number of different triggers for each of its connectors.

  • Automated – triggered when an event happens in the source system.
  • Instant (Button) – triggered by tapping a button on your mobile device or from the Flow portal. Gives the user the ability to manually trigger a Flow to start.
  • Scheduled – triggered based on a schedule defined by the Flow maker.

The following sections describes in more detail the automated triggers in relation to Dynamics 365 integration.

Inbound Integrations

The triggers used for Microsoft Flow to add data into Dynamics 365 depend on the source system and the automated triggers available for the connector of the source system. For example, the SharePoint connector has a trigger called When a file is created or modified in a folder. This trigger can be used for dumping a flat file to trigger integration into Dynamics 365.

Outbound Integrations

Flow uses the automated triggers for the Common Data Service to send data from Dynamics 365 to other systems. Currently for the Common Data Service, there are three of these triggers which are obvious…

  • When a record is created
  • When a record is updated
  • When a record is deleted
Common Data Service Connector and TriggersThree Automated triggers for the Common Data Service connector

But wait there’s more…

There is a fourth trigger for the Common Data Service.

  • When a record is selected

This trigger is an instant trigger and allows the user to manually trigger a Flow to run against a Dynamics 365 (or Common Data Service) record from a button.

Trigger Flow from Dynamics 365 recordFlow button in Dynamics 365.

However, it is difficult to find the When a record is selected trigger when creating a Flow. It does not exist under the instant triggers when creating a Flow from blank. This is where you would expect it to appear. When you click the Instant – from blank option you are asked to search for the instant trigger, but it is not available. If you click Skip then you are able to search for the trigger in the Flow designer page. See the screenshots below.

When a record is selected trigger is missing from Instant Flow menuThe When a record is selected trigger does not appear when creating a new instant Flow as expected.
Skip on CDS Flow connector set up for When a record is selected triggerClick Skip to go to an empty Flow designer page where you will then be able to see the When a record is selected trigger.

Alternatively, you can create the Flow from either the Common Data Service button template or from the Flow button within Dynamics 365.

Common Data Service Button TemplateThe Common Data Service button template uses the When a record is selected trigger.
Create Flow Button in Dynamics 365The Create a flow option under the Flow button in Dynamics 365 opens the Common Data Service button template in Flow.

Working With Dynamics 365 Field Types

Developers who write code for Dynamics 365 understand the different field types within Dynamics 365 or CDS and know that they are not always straight forward to populate from code. Flow is a “Citizen Developer” tool designed for non-developers. However, to create a Flow which reads or updates data from Dynamics 365 you need to understand the different field types.

For example, an Option Set field, such as the Relationship Type field on the Account entity, is shown in the user interface as a list of options. However, behind the scenes, CDS holds an integer value for each option. To update an Option Set field, Flow needs to feed the CDS connector an integer value.

Option Set exampleOption Set example.

Dynamics 365 Field Type Complications

Microsoft have documented a list of the Dynamics 365 field types. However, this list doesn’t explain how to use the field types with Microsoft Flow. The Microsoft Flow documentation for using Flow with the Common Data Service and Dynamics 365 have a couple of sections which give some guidance about the different field types, but still limited. I see this being a big hurdle for Citizen Developers when using Flow to connect to Dynamics 365 or CDS.

The different field types for Dynamics 365 or CDS mean that transformation steps within your Flow are required to ensure the integrated data is compatible between the source and destination systems. This can make the Flow configuration complicated and hard to maintain. Watch out for this.

Microsoft Flow and Dynamics 365 Integration Considerations

This section contains some important considerations when using Microsoft Flow for Dynamics 365 integrations.

Performance

Don’t expect fast integrations with Microsoft Flow. I conducted a test using Microsoft Flow to update 100 records in Dynamics 365. It took over 6 minutes to complete!

There is a way to make Flow faster in this scenario. To loop through a bunch of records or rows etc. an Apply to each action is used. The Concurrency Control setting of this action sets the degree of parallelism (aka the number of rows Flow processes at the same time). I turned this option on, left the degree of parallelism to the default (20) and the results were a lot faster, 35 seconds!

Microsoft Flow run resultsFaster Flow duration time when the Concurrency Control is turned on.
Set Concurrency on Apply to each action in Microsoft FlowChanging the concurrency setting on the Apply to each action

Unnecessary Database Updates

Imagine a scenario where another system is sending data to Dynamics 365, via Flow, to update a Dynamics 365 Account record. The other system sends all fields for the Account regardless of the fields that have changed.

It is best practices not to update the Dynamics 365 database with data that already exists with the same value. Using Flow, it is difficult to achieve this when updating a record in Dynamics 365 without the logic becoming complex and un-maintainable. This is another article in itself but Flow makers must be aware of the implications of writing data to Dynamics 365 that has not changed. For example, what workflows or plugins will it trigger?

Level up Microsoft Flow with Azure Logic Apps

The sections above describe many restrictions on using Flow for integrations, once you dive into it the details. What happens if you have developed a Flow for integration but the integration or Dynamics 365 requirements change such that Flow is no longer suitable?

Upgrade to Azure Logic Apps is the answer. Convert Flows into LogicApps by exporting the Flow and importing into Azure.

You can find a good article on Microsoft Flow vs Azure Logic Apps here. This video from Microsoft Ignite 2018 also gives a simple overview of the difference between Flow and Logic Apps.

CSV File Integration with Dynamics 365 Example

Very soon I will be adding a link to an article which gives a detailed example of how you can build your own CSV file to Dynamics 365 integration using Microsoft Flow. Get an email notification when the article is posted by signing up at the bottom of the page.

Conclusion

For the right use case, I believe that Flow is a great integration option for Dynamics 365 to keep things simple, costs down and maintenance low. However, you do need to consider Flows limitations combined with Dynamics 365 nuances as it can get complex very quickly.

Flow is not a finalised product and Microsoft continue to develop it. I’m looking forward to the future updates announced for Flow to bring it into parity with Dynamics 365 workflow.

The post Dynamics 365 Integration with Microsoft Flow appeared first on Dynamics Citizen Developer.