Hello
I am using Microsoft Fabric (Dataflow gen2 and Notebook with pyspark) to connect to 365 Business central API successfully. The idea is to extract data from source 365 Business central(accounts, customers, employee, etc. ) to target Fabric onelake. The initial Load process is implemented successfully in both two ways. Now I want to implement incremental load. Here is the question/issue we noticed :
- If there are deleted records in source Business Central, how can we detect that thru API?
- Or is there any soft delete flag in Business Central API/ tables, so we can refer that to manually delete the same records in our target Fabric onelake?
If we cannot find a way that auto-detect deleted records in the source Business Central data, INCREMENTAL load will never notice the deleted records. Then FULL refresh for all the tables have to be implemented, which does not make sense for us, especially for the tables have large volume data.
Thanks a lot.