Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Forums / Dynamics 365 general forum / Allow for XML schema d...
Dynamics 365 general forum
Answered

Allow for XML schema drift in ADF Copy Data activity (from Fetch XML)

(0) ShareShare
ReportReport
Posted on by kade.hyland 5

I have observed the same issue as reported in   .
Essentially, when you have a column that is sparsely populated (mostly contains NULLs) from a FetchXML query, the column may not be returned at all by your ADF Copy Data action.  I believe this occurs because fetchXML does not return the attribute tag in the output when the value for the record is null. 

This problem can be resolved by providing a fixed mapping to ADF.  However, my issue is that I'd like to allow for schema drift (e.g. new field added to the source table in Fetch XML) when copying data in ADF and I wondered if there is an alternative solution that would address this requirement?

Categories:
Integration with Other Products
  • kade.hyland Profile Picture
    kade.hyland 5 on at
    RE: Allow for XML schema drift in ADF Copy Data activity (from Fetch XML)

    Thanks for the detailed response Daniele.  I'll have to do some research to determine the best way to collect the metadata for the mapping, but the process you describe makes sense to me and I really appreciate it.

  • Verified answer
    DAnny3211 Profile Picture
    DAnny3211 9,217 Moderator on at
    RE: Allow for XML schema drift in ADF Copy Data activity (from Fetch XML)

    hi

    One possible solution to address this requirement is to use a dynamic mapping approach in ADF. This involves using a combination of dynamic content and metadata-driven mappings to handle schema drift.

    In this approach, you can use dynamic content to extract the schema information from the source dataset and use metadata-driven mappings to dynamically map the columns based on their names and data types. This way, if a new field is added to the source table in FetchXML, it will automatically be included in the mapping without the need for manual updates.

    To implement this approach, you can use the dynamic content feature in ADF to extract the schema information from the source dataset. You can then use this information to dynamically generate the mapping structure using metadata-driven mappings. This can be done using an ADF Data Flow with a metadata-driven mapping transformation, which can dynamically map the columns based on their names and data types.

    Overall, this approach can provide a flexible and scalable solution to handle schema drift in FetchXML queries when copying data in ADF.

    DAniele

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Replay: Dynamics 365 Community Call ERP Edition | October 9, 2024

Another great gathering of our Community!

Anton Venter – Community Spotlight

Kudos to our October Community Star of the month!

Announcing Our 2024 Season 2 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

more Community News

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 290,570 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 228,683 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,148

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans