The views and opinions expressed in this blog are those solely of the author(s) and do not necessarily reflect Microsoft’s current policy, position, or branding. For official announcements and guidance on Dynamics 365 apps and services, please visit the Microsoft Dynamics 365 Blog.
Personalized Community is here!
Quickly customize your community to find the content you seek.
‘Better Together’ Integration forum available
We're launching a how-to forum where you can learn and engage about how Dynamics 365 integrates with other Power Platform products.
Read about Better Together forum
2020 Release Wave 2Discover the latest updates and new features to Dynamics 365 planned through March 2021.
Release overview guides and videos Release Plan | Preview 2020 Release Wave 2 TimelineWatch the 2020 Release Wave 1 virtual launch event
Ace your Dynamics 365 deployment with packaged services delivered by expert consultants. | Explore service offerings
Connect with the ISV success team on the latest roadmap, developer tool for AppSource certification, and ISV community engagements | ISV self-service portal
The FastTrack program is designed to help you accelerate your Dynamics 365 deployment with confidence.
FastTrack Program | Finance TechTalks | Customer Engagement TechTalks | Upcoming TechTalks
In my previous article I shared some general information about Release Waves. How it all works, when they come, how can we prepare etc. I also highlighted some Power Apps features that come with 2020 Release Wave 2. The purpose of this article is to highlight a chapter of the Power Platform Release Plan that contains some Power Platform built-in features for data integration – the chapter with the heading Common Data Model and data integration.
Let me begin with a reminder of what the Common Data Model (CDM) and the Common Data Service (CDS) are. I will just give you this wonderful quote again, from my MVP friend Jonas Rapp and then feel free to dig into the documentation that I just provided you with.
The CDM is like the map and the CDS is the landscape we walk on.Jonas Rapp
The CDM is like the map and the CDS is the landscape we walk on.
The CDM and data integration section in the plan covers new and enhanced features for the following areas:
Dual Write – Dynamics 365 Finance and Supply Chain Management data to the CDSExport to data lake – Data from the CDS to Azure Data Lake Gen 2Power Platform Connectors – The plan covers mostly the certification processPower Platform Dataflows – Load data from different sources to the CDSPower Platform Gateway – Connect to on-premises sourcesPower Query Online – For data preparation and transformation
The highlights presented comes from the chapters about Export to Data Lake, Power Platform Connectors, Power Platform Dataflows and Power Query Online. Please note that here are more new features to read about in the plan, just a few are presented here!
There are a lot of features presented for Export to data lake. A few highlights are: Cross-tenant support for when exporting to data lake. Until this wave the Azure data lake gen 2 storage had to be in the same tenant as the CDS. Support for exporting entity Audit data to Azure data lake storage gen 2. Support for exporting attachments – support for the annotation entity. Support for soft delete – it will be possible to delete the data from the source but keep it in the data lake for analysis purpose.
For Power Platform Connectors you can read that partners creating and certifying connectors will get feedback for certified connectors, enhanced version management and we will get more open source connectors from Microsoft to build upon or contribute to. We find these on GitHub, take a look here! The plan also presents that we will get better guidance for when connectors or operation are deprecated, advice for what to use instead. Read more about the certification process here.
What is new for Power Platform Dataflows? First of all, do not forget to read the Power Query Online part – these new features will be applicable for Power Platform Dataflows as well. New administration capabilities are presented. Until now it has been the creator of a dataflow which has been able to see and edit the dataflow and we have had no possibility to change the owner of a dataflow. New administration capabilities are coming to us, we will be able to view all dataflows created in environments and also to reassign dataflows to new owners. The plan also mentions we will be able to “certify” dataflows. Exactly what that means is yet to be found out.
Power Platform Dataflows grow up into Azure Data Factory. We will get an extension possibility – take a Power Platform Dataflow and extend it into an Azure Data Factory Data Flow. This is presented as taking the Power Platform Dataflows from a Citizen developer approach to an ETL Pro developer approach with Azure Data Factory Data Flows.
Read more about Azure Data Factory in the Microsoft Docs and take a look at how to create an Azure Data Factory Data Flow here. You can also read the Azure Blog post announcing GA for ADF Mapping Data Flows here and you can find a lot of other great links here.
I am interested in hearing your thoughts about in what scenarios we might want to extend a Power Platform Dataflow into using Azure Data Factory Data Flows. Please let me know your thoughts!
Here follows some examples of what is new for Power Query Online. Certified connectors (developed by partners) is on the way to Power Query Online. Read more about the different connectors here. The possibility to use Custom Connectors will come to Power Query Online. If you are interested in how to get started with Custom Connectors for Power Query, take a look here. Please note, these are not the same as the connectors we work with when we use Canvas Apps and Power Automate flows.
As already mentioned, the features presented under Power Query Online is applicable also for Power Platform Dataflows. So we will get more possibilities to use new data sources, that is how I read the plan, since we will get certified connectors and Custom Connectors possibilities. Templates are coming. Import/export Custom Dataflow templates and to define parameters in the templates. Reuse Dataflow definitions in different environments. This feature has been available in Power Query desktop, read about it here. Other interesting things you can read about is a future possibility to enrich data from data sources such as Azure Maps to get smart data transformation, e.g address geocoding.
One thing that crosses my mind when I go through the plan is that there is nothing in it about the “Data Integrator“. The Data Export Service is not mentioned either.
That was the highlights for the CDM and data integration section of the 2020 Release Wave 2 plan. I hope you are looking forward to the new features and can’t wait to try them out!
Business Applications communities