web
You’re offline. This is a read only version of the page.
close
Skip to main content
Community site session details

Community site session details

Session Id :
Dynamics 365 Community / Forums / FastTrack for Dynamics 365 forum / Export Dataverse Data ...
FastTrack for Dynamics 365 forum
Unanswered

Export Dataverse Data into Azure SQL using Synapse Pipeline

(1) ShareShare
ReportReport
Posted on by Anvesh Ananthula 9

I'm working on to set up Azure Synapse pipline template DataverseToSQL. I am following the Microsoft documentation:  https://docs.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-pipelines and there's seems to be an issue with step 4 on Blob path ends with: /model.json . I'm getting 0 files matched in my blob. Instead of Blob path ends with, i have added model.json in blob path begins with and it showed model.json in below window. Issue with Blob path ends with.

pastedimage1667405367410v1.png

Is this supposed to be like this ?

Issue 2 :  Observed pipeline is triggered every one hour and each individual pipeline is running more than 15 hrs. I'm thinking the above pipeline loads entire data instead of incremental data. Is there any setting in the pipeline to take only incremental/delta changes to process. 

We have more than 10 million records in one entity. 

pastedimage1667405672300v2.png

Can you please help me to resolve above two issues.

I have the same question (0)
  • Bryan Tuttle Profile Picture
    Bryan Tuttle 22 on at
    RE: Export Dataverse Data into Azure SQL using Synapse Pipeline

    Hi Anvesh,

    I too am attempting to get this to work but stuck on the Trigger failing. However I am able to get this to work if I hard code both the Container and the File info. You could test the same way I am which is to try is to test the hardcoded values for the container and the file in place of the two parameters in the trigger run parameters.  In the document it is not clear what\which Model.Json should be used. The model.json file at the root container or the file within the incremental folder...

    So check the container if you are only seeing a single folder then you have not setup the incremental option (which is quite the hack, you need to change the URL with a Query String Param when setting up/using the Azure Data Link (its right at the list of Tables you wish to Sync to data link). So if you missed the ADVANCED drop down you will end up with a different folder structure than the article expects.

    Also check that you have Published everything. 

  • Anisur Profile Picture
    Anisur 19 on at
    RE: Export Dataverse Data into Azure SQL using Synapse Pipeline

    HI  Anvesh Ananthula

    Blob path ends with : /model.json  (Should be)

    Blob path ends with : \model.json  (Should  not be)

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

News and Announcements

Join experts in person to explore low code and AI agents at the Power Platform Community Conference sponsored by Microsoft

Quick Links

Responsible AI policies

As AI tools become more common, we’re introducing a Responsible AI Use…

Abhilash Warrier – Community Spotlight

We are honored to recognize Abhilash Warrier as our Community Spotlight honoree for…

Congratulations to the September Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > FastTrack for Dynamics 365

#1
CA Neeraj Kumar Profile Picture

CA Neeraj Kumar 17

#2
MH-12092052-0 Profile Picture

MH-12092052-0 12

#3
AS-12100900-0 Profile Picture

AS-12100900-0 4

Last 30 days Overall leaderboard

Featured topics

Product updates

Dynamics 365 release plans