Microsoft Dynamics 365 | Integration, Dataverse...

Data Export Services (DES) - Deprecation - Metadata Information

(0) Share

Report

Posted on by Shaun W Harvey

242

Hi,

We have been using Data Export Services (DES) to replicate Dataverse Data to an Azure SQL Database.

DES is being deprecated in November 2022 in favour of Azure Synapse Link For Dataverse.

Do more with data – From Data Export Service to Azure Synapse Link for Dataverse | Microsoft Power Apps

As well using the Azure SQL Database for reporting (as many other people probably do).

We also use the Azure SQL DB as part of a Caching Solution so as to reduce the number of API calls to the Dataverse.

Even with DES not everything that was needed was replicated down. So, we have a scheduled task which retrieves extra infomation (such as Global OptionSet Values) into a custom table we have created in the Azure SQL DB.

We have had a quick look at Azure Synapse Link and there does not seem to be the ability to create our own tables. Probably as they are Data Lake tables rather than an actual SQL instance.

With Azure Synapse Link, Is there a way to still get the data to replicate to an Azure SQL DB where we can easily add our own data if needed?

Best Regards,

Shaun Harvey

I have the same question (0)

All responses (51)

Answers (0)

Jon Meyer 70 User Group Leader on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Shaun,

It sounds like we have a similar use case with our existing DES implementation to your implementation.

We are working now to use Synapse Pipelines to take the data from Synapse and put it back into the Azure SQL database that is currently the target for DES.

Right now, we are waiting on Microsoft to roll-out two items in their April release:

- Incremental updates to Synapse (independent CSVs containing just new updates vs. entire tables)

- Better support for Option Sets

Jon

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Hello, does this mean that the Data Export Service will cease to function (or exist) from Nov 2022 OR the service will continue to remain operational but without any support?

Thanks

Nakul Ringshia

Was this reply helpful? Yes No
Shaun W Harvey 242 on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

docs.microsoft.com/.../important-changes-coming

According to the Deprecation Notice.

"Data Export Service will continue to work and will be fully supported until it reaches end-of-support and end-of-life in November 2022."

I read "End OF Life" as it is being switched off in November 2022.

Was this reply helpful? Yes No
Jon Meyer 70 User Group Leader on at

Like (1)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

We've been actively conducting POCs of alternatives to recreate DES functionality using Synapse Link, Synapse Pipelines, and other approaches like direct file reads of the CSVs and using the JSON metadata to map ADF pipelines on the fly. Nothing works perfectly to recreate what DES does today.

We've written a formal letter to Microsoft leadership asking for a real demonstration of how the new solution can recreate what DES does at scale (i.e., lot of changes in Dynamics to lots of entities / production volume) or to postpone the termination of DES until they can.

I know there is new incremental Synapse Link functionality to be delivered in April. We're waiting to see what this does and how it should be used.

If anyone wants to COLLABORATE on these POCs, please let me know.

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Thanks Shaun & Jon for your responses.

Jon, Could you let me know when was a formal letter written to MS Leadership and any expected response date?

My team is starting to explore the Synapse Link functionality. Based on what I have read so far in this thread, it seems like Synapse Link is not an apples to apples in terms of DES functionality.

Can you please provide "getting started resources" that could help my team? Once we have a decent idea about the functionality, I am sure we can collaborate POCs.

Thanks

Nakul Ringshia

Was this reply helpful? Yes No
Andreas Cieslik 9,267 on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Hi Jon,

I tried to contact you via PM, but don't know if it reached you.

I would like collaborate with you and exchange experiences.

Maybe we can connect via Linkedin?

Thanks,

Andreas

Was this reply helpful? Yes No
Jon Meyer 70 User Group Leader on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Andreas, I believe our PM has been in contact with you? If so, we look forward to the meeting.

Jon

Was this reply helpful? Yes No
Jon Meyer 70 User Group Leader on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Nakul,

If you can give me your contact info (jon.meyer@captrust.com), I will have our PM reach out to share our latest progress.

Jon

Was this reply helpful? Yes No
WABGOR_Dave 70 on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

It seems we're all in the same boat, and that MSFT (or more than likely marketing/sales within the Azure Synapse department) determined they needed another revenue stream. I tried to use the analogy, when explaining to our execs why I may need some added budget for added services, that MSFT decided to redirect traffic the long way around to our destination, via toll roads, that may get to the same spot, and tear up the perfectly good road that exists.

While our requirement doesn't sound anywhere near as complicated/enriched as everyone here, so far, it will impact any new service/product launch. I'll keep an eye on this thread to see if anyone makes any headway. At the moment I may look at having to do a reverse of what we do now; meaning, importing data into CRM, and having sales input that data via an app instead of CRM.

Was this reply helpful? Yes No
Jon Meyer 70 User Group Leader on at

Like (0)

Report

RE: Data Export Services (DES) - Deprecation - Metadata Information

Dave, we have the exact same thoughts. It's got to be about pushing Azure & Synapse (and attendant revenue) and making sure every possible consumption of data from Dynamics CE is related to some licensable event.

Was this reply helpful? Yes No