Hello All,
I have a custom entity for which I need to set up a recurring job that would export data into a DB on daily basis.
For this, every day when the job is run, I only need the delta records to be pushed into the target DB. However, when I try to run the incremental push, all the records get loaded into the staging table(this creates duplicate records in staging), although, since the change tracking is enabled, the target DB does not reflect any duplicate records.
Since all data gets pushed into the staging table, and the volume of data is huge, I want to only bring in the delta records onto staging, so that the data push happens clear cut.
The staging table does have an index that includes a composite key of Def Group, execution id, and 3 other fields that are unique to the record.
Is there any solution to avoid the data duplication in the staging table in D365 FO?
