SharePoint hosted file retrieved by Import Bank Statements batch job sometimes blank

(4) Share

Report

Posted on by Dos

We have set up Electronic reporting on our cloud based Dynamics F&O server so that retrieve text files (in MT940 format) from SharePoint. We have a Power Automate flow that creates these text files. Occasionally, the batch job runs and produces a error reporting the cause to be that the file is blank. The file that is moved into the Error folder in SharePoint is blank. However, the Power Automate flows shows that in every case that it successfully populated the same file with data.

What we think might be happening (due to the low occurrence rate) is that the Dynamics batch job is accessing the file just as the Power Automate has created the SharePoint file but has yet to populate it. Power Automate isn't reporting an error as it still has a handle on the original file. Dynamics is allowed access to the file because SharePoint hasn't locked the file for editing by Power Automate. We think that next the batch job has creates a new copy of the file (the blank version which it thinks is the correct state) in the Error folder. Power Automate may now be updating the file with data but it is updating the file that Dynamics are moved (to the bin?)

Has anyone else experienced something like this?

We would rather avoid introducing a complex scheduling routing to ensure that the files are generated in one cycle and retrieved in another but may be something we resort to. The other element we are currently testing is whether the Power Automate SharePoint Copy file functionality is different from the Create file functionality (i.e. the file only appears fully populated every time).

Any help or ideas of how to avoid this blank file error would be most appreciated.

Thanks

Dominic

Categories:

Dynamics 365 Finance

I have the same question (0)

All responses (5)

Answers (2)

Sort by

Verified answer

Sohaib Cheema 49,668 Super User 2026 Season 1 on at

Like (3)

Report
Copy link

Link copied!

Hi @Dos

I haven’t faced the exact same issue, but I’ve come across a few others along similar lines:

Once in AX 2009, We had an issue where, occasionally, the files being produced had a duplicate name (same as a previous file name). We had to customize the code in the standard class, to include milliseconds and a few other elements to ensure the filename was unique every time.

I also recently encountered an issue where Dynamics 365 created duplicate files in SharePoint instead of just one (using Electronic Reporting classes). That issue is still open with Microsoft it's producing two files every time a process runs.

For your problem, could you introduce a staging folder where Power Automate initially creates the file with a naming convention like TempDateYYMMDDHourMinuteSecondMillisecondsRandomBits...? Then, Power Automate populates the data in the file, and once that’s complete, it renames the file by removing Temp from the name. Finally, it moves the file to another folder (say, ReadyForProcessing).

Dynamics can then be configured to pick up files only from the ReadyForProcessing folder.

1 people found this reply helpful.

Was this reply helpful? Yes No
André Arnaud de Cal... 305,643 Super User 2026 Season 1 on at

Like (2)

Report
Copy link

Link copied!

Hi Dominic,

Sohaib already suggested renaming or copying the file after completion. This can be a solution. However, we don't know the details of the Power Automate flow and the recurrence settings. You can also review the recurrence and moments when the flow is running and when Dynamics 365 will poll for a particular file. If you e.g. create the text file at 8 AM and processes them at 9 AM, there shouldn't be a conflict, unless the creation of the file takes more than an hour.

1 people found this reply helpful.

Was this reply helpful? Yes No
Suggested answer

Dos 17 on at

Like (0)

Report
Copy link

Link copied!

Hi @Sohaib Cheema and @André Arnaud de Calavon,

Thank you both for your prompt replies. Your suggestions are all valid options and it's reassuring that we have been considering the same options to implement.

Staging folder: As mentioned, for the moment we've chosen to use the Copy file technique (similar to the Move file technique mentioned) where we create the file in a different location from where Dynamics is monitoring (and then copy / move it to the monitored folder afterwards). I will update this ticket if we still experience the same issue (after obviously using the Move option first).

Random file name: The files we are creating already use the time (to milliseconds) in an attempt to use unique file names (it's reassuring that others use the same technique).

File rename: That is a further option we had considered. If the staging folder option still produces errors we plan to change the Dynamics batch job to exclude files beginning with "temp". Then we will change the Power Automate flow to Create and Move files beginning with "temp". Then, as a final step, remove the "temp" element of the filename so that Dynamics can "see" it. Depending on how Dynamics determines whether it has "seen" a file before, it is possible that this solution may introduce a new issue (where it ignores these renamed files as it thinks it has "gone past them"). I doubt that this is the case, but I'm mentioning it here for my potential future reference just in case.

Flow Scheduling: As mentioned in my original post, we have considered "complex" scheduling (i.e. a window of time to create files and a different window of time for importing / reading them). There are other elements to consider with this solution, like as you suggested, if one of the windows overruns as thus overlaps the other. This is difficult to control as the scheduling is controlled by two different machines (Dynamics and Power Automate) and you do not have the normal scheduling tools that would stop overlapping occurring. Also, without thorough documentation (which every organisation creates and uses of course) the point of the scheduling may be lost when I retire. This option is still worth considering though, but I believe that we will leave this as our final back-up plan to use if the other options do not. So again, a very worthy suggestion.

Thanks again for your valuable feedback

Dominic

Was this reply helpful? Yes No
Verified answer

Sohaib Cheema 49,668 Super User 2026 Season 1 on at

Like (1)

Report
Copy link

Link copied!
Hi @Dos

At many implementations, I see a three-folder structure:

IN Folder – Ready for Dynamics to process

Processed Folder – Files that Dynamics has successfully processed

Failures Folder – Files that Dynamics failed to process

I’m suggesting a fourth folder called the Staging Folder, which would serve as the main folder for Power Automate.

Power Automate should create the file in the Staging folder, populate it with data while it remains in that folder, and once complete, move the file to the IN folder (the first folder). Dynamics is not aware of the Staging folder at any point.

1 people found this reply helpful.

Was this reply helpful? Yes No
Suggested answer

Cyrille Nembot 151 on at

Like (0)

Report
Copy link

Link copied!

Of
course. Based on your description, you are almost certainly
experiencing a classic race condition where the Dynamics 365 batch job
is polling the SharePoint folder and finding the file at the precise
moment it has been created by Power Automate but before the data has
been written, resulting in the processing of a blank file. The most
robust solution to this is to implement a handshake mechanism, such as
the marker file pattern, to eliminate the timing issue entirely. This
approach involves modifying your Power Automate flow to first write the
complete file to a temporary staging folder or using a temporary file
extension; only after the data is fully written would the flow then move
the file to the final monitored directory or rename it to its proper
extension. This ensures the file only becomes visible to the Dynamics
365 ER process once it is completely populated, preventing the batch job
from ever accessing a partial or blank file. While a simple delay in
the batch job could serve as a temporary mitigation, it is a fragile
solution, whereas re-engineering the file creation process to be atomic
is a definitive fix that avoids the inherent unreliability of trying to
coordinate timing between two independent automated systems.

Was this reply helpful? Yes No