We're using F&O Export to Datalake to have near real-time access to F&O data in Synapse Datalake. Both for reporting, and integrations.
However any file currently being updated, errors when queried. This is a known issue per documentation:
MS Documentation
/
Q: When I read data files by using Synapse serverless, I notice an intermittent error: /Unexpected end-of-input within record at..../
A: When tables records are frequently updated or deleted in finance and operations apps, you might notice a read/write contention on CSV files, and this error might occur. This issue occurs when the Synapse serverless SQL service queries a CSV file as it's being updated. In many cases, you might be able to fix this issue by retrying the query.
/
Not a good user experience.
How have you dealt with this?
The only two things I can think of are:
1.) One can enable ALLOW_INCONSISTENT_READS in their serverless database. But, we were already doing that.
2.) Switch to using CETAS. But, those can only be destroyed + rebuild. A tast best suited at night and....now we no longer have that 'near real time' advantage, it's nightly. Kind of defeats the purpose of 'export to datalkae' over our old BYOD.
158
Share
Report
All responses (
Answers (
