I have a business need where I need to be able to create several thousand records when one single record is created. These records are based on a hierarchy of accounts. Ie when a custom entity is created on the top account it needs to trigger and create a duplicate record for all the accounts under that account in the account hierarchy (literally "under" not just children but all descendants). I need to be able to process at least 7000 records. I have tried the following:
- I created a plugin but it hit the 2 minute timeout - (Query all accounts under and loop through creating records)
- I created a cascading down plugin but it still hit the 2 minute timeout because we have certain accounts that have several hundred direct children - (Query all direct children and loop through creating records that then triggers creating records for all their children all the way down the hierarchy)
- I created a cascading down and crawling across plugin but it hit the 8 depth and was classified as an infinite loop - (This was complicated but worked on small scale, and like everything else failed when scaled up)
- I created a Microsoft Flow job that creates the records for all the children and then restarts on the create of the children. I got this to work about half the time, but Flow seems to have issues with not always triggering and sometimes triggering twice creating duplicates. Almost every other time when I run on the large record sets it either doesn't trigger or runs and creates too many records. The other half of the time it works fine. I am working with the same set of data so it is not likely a problem with the flow.
Any ideas on how I could address this problem? I am running out of ideas. I am considering am Azure Function or Logic App, but I have never worked with them and not sure what there capabilities are.