Hi Antonis - Glad we were able to assist you
As long as there are no errors (like database table locking) encountered by your AL Code (I am assuming it will be a Job queue) I do not see it can be of any issue per-se.
However, I'd suggest grab a batch of 100 files at once and then process these 100 records in your table first before moving to the next 100. This way loop your solution 10 times to see how the performance is - In my experience, you may not get a lot of errors but if things affect performance, it is as good as an error (or even worse!)
1. Keep a provision (a setup based field) such that you can increase/decrease this "batch size" as you want it
2. Check with your Azure Admin if it has enough capacity to deal with such a workload
3. Consider re-evaluating/optimizing your code and preventing roll backs in case data is written into high-volume tables (Sales Orders, Purchase Orders, etc.) - basically wherever the users may visit the most during the day is your high-volume table. If your custom code involves "posting" of any document, then you especially need to take care about the transaction table locking issue (if the posting will be undertaken during users working hours)
4. Try running your Job Queue (Which grabs the files from Azure and puts in BC) during times either when the user traction is minimal on BC or after office hours. This may not be possible in all scenarios, given when you receive the data
Lastly, it can be a bit time consuming but if you're really after making this scale-up dynamically you need to monitor where it lags - try implementing telemetry for your custom app. This way if something's taking too long, you'll get to know:
learn.microsoft.com/.../telemetry-overview
P.S: Optimizing your code can mean one or more of the below activities (non-exhaustive list)
1. Sort wherever necessary (If you know the recently created data is what you're after, sort DESCENDING on the created by D/T field)
2. Use filters tightly (If you know you want to grab Document Type = Purchase Receipt & Entry Type = Purchase, make sure to write them down in your SETRANGE, although it may seem obvious that they both would appear together)
3. Use temporary tables wherever you are intense on looping &/or validating stuff
4. Introduce error handling (You'd absolutely hate it if you were made to manually monitor the JQ everyday to see if there were errors or it was successful. Use error handling capabilities in AL to let your JQ function even if there is an error)
Best of luck :-)
