You’re offline. This is a read only version of the page.
5
I have observed the same issue as reported in .
Essentially, when you have a column that is sparsely populated (mostly contains NULLs) from a FetchXML query, the column may not be returned at all by your ADF Copy Data action. I believe this occurs because fetchXML does not return the attribute tag in the output when the value for the record is null.
This problem can be resolved by providing a fixed mapping to ADF. However, my issue is that I'd like to allow for schema drift (e.g. new field added to the source table in Fetch XML) when copying data in ADF and I wondered if there is an alternative solution that would address this requirement?
Thanks for the detailed response Daniele. I'll have to do some research to determine the best way to collect the metadata for the mapping, but the process you describe makes sense to me and I really appreciate it.
hi
One possible solution to address this requirement is to use a dynamic mapping approach in ADF. This involves using a combination of dynamic content and metadata-driven mappings to handle schema drift.
In this approach, you can use dynamic content to extract the schema information from the source dataset and use metadata-driven mappings to dynamically map the columns based on their names and data types. This way, if a new field is added to the source table in FetchXML, it will automatically be included in the mapping without the need for manual updates.
To implement this approach, you can use the dynamic content feature in ADF to extract the schema information from the source dataset. You can then use this information to dynamically generate the mapping structure using metadata-driven mappings. This can be done using an ADF Data Flow with a metadata-driven mapping transformation, which can dynamically map the columns based on their names and data types.
Overall, this approach can provide a flexible and scalable solution to handle schema drift in FetchXML queries when copying data in ADF.
DAniele
Stay up to date on forum activity by subscribing. You can also customize your in-app and email Notification settings across all subscriptions.
#1
André Arnaud de Cal...
291,253
Super User 2024 Season 2
#2
Martin Dráb
230,188
Most Valuable Professional
#3
nmaenpaa
101,156
Share
Report
All responses (
Answers (