Finance | Project Operations, Human Resources, ...

Failed to insert record into staging table. Duplicate records must be removed from the file

(2) Share

Report

Posted on by ..

1,915

Hi,

When i import 1 record at a time, i get no errors. However, when i import more than one, i get this error below.
So if i have new 4 records to import, if i import them 1 by 1 it works, if i import the 4 records together in the excel, i get this error

I'm using entity SalesOrderLineV2Entity, and there is a customization on it where i added an outer join datasource with salesLine.

Failed to insert record into staging table. The keys of the record are DEFINITIONGROUP, EXECUTIONID, INVENTORYLOTID, PARTITION. Duplicate records must be removed from the file prior to import.
<?xml version="1.0"?><Errors><Error><ErrorCode>-1071636471</ErrorCode><SubComponent>OLE DB Destination [526]</SubComponent><Description>SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80004005  Description: "The statement has been terminated.".
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80004005  Description: "Violation of PRIMARY KEY constraint 'I_18374STAGINGIDX'. Cannot insert duplicate key in object 'dbo.SALESORDERLINEV2STAGING'. The duplicate key value is (5637144576, import sales line v2, import sales line v2-2025-01-13T00:35:23-B8EA12CAB8EC4098BED3327A437E9854, ).".
</Description></Error><Error><ErrorCode>-1071607767</ErrorCode><SubComponent>OLE DB Destination [526]</SubComponent><Description>SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "OLE DB Destination.Inputs[OLE DB Destination Input]" failed because error code 0xC020907B occurred, and the error row disposition on "OLE DB Destination.Inputs[OLE DB Destination Input]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.
</Description></Error><Error><ErrorCode>-1073450974</ErrorCode><SubComponent>SSIS.Pipeline</SubComponent><Description>SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "OLE DB Destination" (526) failed with error code 0xC0209029 while processing input "OLE DB Destination Input" (539). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.
</Description></Error></Errors>

Categories:

Dynamics 365 Finance

I have the same question (0)

All responses (7)

Answers (0)

Suggested answer

Komi Siabi 13,049 Most Valuable Professional on at

Like (0)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

The excel sheet definitely has some blank rows being imported into Dynamics.

Delete all rows after the 4 records, and re-import. Also ensure there are not duplicate INVENTORYLOTID on the file.

Was this reply helpful? Yes No
.. 1,915 on at

Like (0)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

Hi Komi,

I exported a sales order with 4 lines, then i deleted the order lines from SQL

then i opened the exported file, and deleted the values under InventoryLOTID -- i clicked save, closed the file then imported it again. still same error

Was this reply helpful? Yes No
Suggested answer

Alireza Eshaghzadeh 14,637 Super User 2025 Season 2 on at

Like (1)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

Hi,

Based on the error message, it appears there is a duplicate in the source data for the entity's key fields. Please verify that the values in the field "InventoryLotId" are unique. Duplicates in this field will cause the issue, so ensuring uniqueness should resolve it.

Was this reply helpful? Yes No
.. 1,915 on at

Like (0)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

Hi Alireza,

I mentioned that i made the invenotryLot values as empty in the excel in order to consider it as an insert.

if i import more than one record it fails, if i import only one it works.

I tried in another environment where the customization is not there but same error

Was this reply helpful? Yes No
S Gopi Krishna 579 on at

Like (0)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

Try removing the inventory lot id column from the excel sheet and import.

Was this reply helpful? Yes No
DN-27070130-0 61 on at

Like (0)

Report
Failed to insert record into staging table. Duplicate records must be removed from the file

I have a same issue

I want upload Return Order Lines

When I check the mapping field there is No Field / Column below

How I can upload more than 1 rows?

DEFINITIONGROUP, EXECUTIONID, INVENTORYLOTID, PARTITION

Was this reply helpful? Yes No
Suggested answer

Saalim Ansari 648 on at

Like (0)

Report

Failed to insert record into staging table. Duplicate records must be removed from the file

Hii,

You are receiving the error because the staging table's primary key constraint is being violated during bulk import. The staging table for the SalesOrderLineV2Entity includes key columns such as PARTITION, DEFINITIONGROUP, EXECUTIONID, and INVENTORYLOTID. When you import one record at a time, the combination of these key fields remains unique, so the import is successful. However, when you try to import multiple records together and the INVENTORYLOTID is either blank or null in all rows (which is likely due to the outer join you added on SalesLine), the same key combination is repeated, causing a duplicate key error.

To resolve this issue, you need to ensure that the INVENTORYLOTID or whichever field is causing the duplication has a unique value for each record. One way to do this is to update your Excel file to assign a unique value (like a row number or identifier) in a column that maps to a key field in the entity. Alternatively, you can fix this in your customization by making sure the join with SalesLine doesn’t result in null values, or by programmatically assigning a unique temporary value to the INVENTORYLOTID in your data processing logic. Avoid modifying the staging table’s primary key unless absolutely necessary, as it may break the standard data management functionality. Ensuring uniqueness in the key fields is the safest and recommended approach to avoid such errors during bulk import.

Was this reply helpful? Yes No