web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

No record found.

News and Announcements icon
Community site session details

Community site session details

Session Id :
Microsoft Dynamics CRM (Archived)

Import CSV File via Console App Not Processing correct number of records

(0) ShareShare
ReportReport
Posted on by

Hey Guys,

Looking for some assistance if possible. First time developing with Dynamics and I have written a console app to process records from a csv file and enter them into Dynamics CRM. The code I have written seems to process all the records fine but when I look at the records in CRM once the console app has ran the total records do not equal to the records in the csv file.

The record set I am processing is around 15-20K and I am using the ExportMultipleRequest to process the batches in batches of 1000 due to the maxbatchsize being 1000 by default. Here is my code any help to diagnose the issue would be greatly appreciated.

  public static void InsertPlanSales()
        {
            // Get the CRM connection string
            CrmServiceClient crmConn = new CrmServiceClient(ConfigurationManager.ConnectionStrings["CRM"].ConnectionString);
            IOrganizationService crmService = crmConn.OrganizationServiceProxy;
            EntityCollection planSalesList = new EntityCollection();

            // Create an ExecuteMultipleRequest object.
            ExecuteMultipleRequest requestWithResults = new ExecuteMultipleRequest()
            {
                // Assign settings that define execution behavior: continue on error, return responses. 
                Settings = new ExecuteMultipleSettings()
                {
                    ContinueOnError = false,
                    ReturnResponses = true
                },
                // Create an empty organization request collection.
                Requests = new OrganizationRequestCollection()
            };

            try
            {
                string csvPath = ConfigurationManager.AppSettings["localFile"];

                //Read the contents of CSV file.
                var csvRows = File.ReadAllLines(csvPath);

                //Start at 2nd row as first is headers
                int startRow = 1;

                //Loop through the rows in the CSV.
                for (int i = startRow; i < csvRows.Length; i++)
                {
                    string currentRow = csvRows[i].Replace("\"", string.Empty);
                    string[] columnValues = currentRow.Split(',');
                    Entity planSaleDetails = new Entity("red_sales");

                    //Map values to Dynamics fields.
                    planSaleDetails.Attributes["red_anubisnumber"] = columnValues[0];
                    planSaleDetails.Attributes["red_allocationnumber"] = columnValues[1];
                    planSaleDetails.Attributes["new_agenttype"] = columnValues[2];
                    planSaleDetails.Attributes["red_fdtype"] = columnValues[3];
                    planSaleDetails.Attributes["new_sellername"] = columnValues[6];
                    planSaleDetails.Attributes["red_sellerhoname"] = columnValues[7];
                    planSaleDetails.Attributes["red_fdname"] = columnValues[8];
                    planSaleDetails.Attributes["red_fdhoname"] = columnValues[9];
                    planSaleDetails.Attributes["red_plannumber"] = columnValues[10];
                    planSaleDetails.Attributes["red_brander"] = columnValues[11];
                    planSaleDetails.Attributes["red_applicationtype"] = columnValues[12];
                    planSaleDetails.Attributes["red_paymenttype"] = columnValues[13];
                    planSaleDetails.Attributes["red_salestatus"] = columnValues[14];
                    planSaleDetails.Attributes["red_applicationdate"] = columnValues[15] == string.Empty ? null : (DateTime?)Convert.ToDateTime(columnValues[15]).ToUniversalTime();
                    planSaleDetails.Attributes["red_startdate"] = columnValues[16] == string.Empty ? null : (DateTime?)Convert.ToDateTime(columnValues[16]).ToUniversalTime();
                    planSaleDetails.Attributes["red_cancellationdate"] = columnValues[17] == string.Empty ? null : (DateTime?)Convert.ToDateTime(columnValues[17]).ToUniversalTime();
                    planSaleDetails.Attributes["red_scorecardchannel"] = columnValues[18];
                    planSaleDetails.Attributes["new_salesvalueatpos"] = columnValues[19] == string.Empty ? null : (Double?)Convert.ToDouble(columnValues[19]);
                    planSaleDetails.Attributes["red_salesvalueatalloc"] = columnValues[20] == string.Empty ? null : (Double?)Convert.ToDouble(columnValues[20]);
                    planSaleDetails.Attributes["red_salestype"] = columnValues[21];
                    planSaleDetails.Attributes["red_maturitystatus"] = columnValues[22];
                    planSaleDetails.Attributes["red_plancategory"] = columnValues[23];

                    planSalesList.Entities.Add(planSaleDetails);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }

            int remainingRecords = planSalesList.Entities.Count();

            foreach (var entity in planSalesList.Entities)
            { 
                CreateRequest createRequest = new CreateRequest { Target = entity };
                requestWithResults.Requests.Add(createRequest);
                remainingRecords--;
                if (requestWithResults.Requests.Count == 1000 || remainingRecords == 0)
                {

                    var responseWithResults = crmService.Execute(requestWithResults) as ExecuteMultipleResponse;
                    requestWithResults.Requests.Clear();
                    Console.WriteLine("{0}.{1}.{2}", "1000 Records Processed", "Remaining Records ", remainingRecords );
                }

            }
 
        }

*This post is locked for comments

I have the same question (0)
  • Suggested answer
    rthompson Profile Picture
    1,532 on at

    Hi Ichigo65,

    Looks like you just need to change this planSalesList.Entities.Add(planSaleDetails) not need to create a collection to write out.

    to

    crmService.Create(planSaleDetails);

    Also if you are creating detail entities and you have a header entity  you will need to create a EntityReference from the header entity.

  • Suggested answer
    rthompson Profile Picture
    1,532 on at

    Plus under Data Management you can use the import process that will work great for what you are doing.  No need for creating a console program for this.

  • Community Member Profile Picture
    on at

    Hi rThompson,

    Really appreciate your feedback. The reason I was looping through the csv records and adding them to a collection was because I tried to process them individually and the performance was terrible. So I looked at the Microsoft Dynamics samples and it said to use the ExecuteMultipeRequest to process batches in 1000 which would speed things up. When I try and process it the code runs successfully but say I have 15000 records in the csv when I check Dynamics after the process ran only something like 14200 are there.

    Also when I loop through the csv file I ignore the first row which is the header row and then I map the values to the specified attribute in Dynamics, is this not how it should be done? DO I need to create an Entity Reference even though I am not using the header row in the CSV file?

    The reason I am using a console app for this purpose is I am automating the import of these records from our ERP system so that it can be an automatic process not a manual one.

    Any help you could provide would be awesome. Thanks again for responding look forward to your feedback

    Ichigo65

  • Suggested answer
    rthompson Profile Picture
    1,532 on at

    Hi Ichigo65,

    Just my suggestion. 

    I know you have spend a lot of time writing your own import program.   For me I try to find the easier and quicker way to import data.

    CRM already has an import process that you can use to do scheduling.  The import process will allow you to map the csv file to the entity.  It's very very easy. 

    Some people will also tell you to use SSIS to do what you are doing. 

    Since you are new at this.  It might be better using the CRM import tool under Data Management.  For what you are doing its much faster and easier this way.

    You can load data and delete data.

    Also,  if you don't have a header entity for this detail entity.  You do not need to create a EntityReference.

    3833.Import.JPG

  • Community Member Profile Picture
    on at

    Hi rThompson,

    Thanks again for your helpful replies. The reason I have to make a console app is because this application will not only import records but will then update them daily in CRM. So the application will insert records but also then run updates on specific attributes in the data in the table if the records contain the specified criteria in my code. Manually doing the import like you mention works and it imports the correct recordset number but when I run my application it does not insert the correct amount. Any ideas why this is causing this?

    Ichigo65

  • rthompson Profile Picture
    1,532 on at

    Question.  Are you only creating new records or are you also updating records?

  • Suggested answer
    Adrian Begovich Profile Picture
    1,032 Moderator on at

    Hi Ichigo65,

    I recommend using SQL Server Integration Services (SSIS) with the KingswaySoft SSIS Integration Toolkit for Microsoft Dynamics 365 for this task.

  • Community Member Profile Picture
    on at

    Creating new records when a sale has taken place and then updating the records if attibute values have changed.

    Ichigo65

  • Community Member Profile Picture
    on at

    Hi Adrian,

    My compnay would prefer our development team handling this inhouse without the use of external commercial toolkits. All I am trying to find out is the cause for the reocrds I am inserting not totalling to the reocrd count from the csv file. My code works and if I step through it in Visual Studio I see it processing all the records so I am trying to see what is causing this issue.

    Any help would be appreciated.

    Ichigo65

  • Suggested answer
    RaviKashyap Profile Picture
    55,410 Moderator on at

    Hi,

    Take a look at the sample-docs.microsoft.com/.../sample-execute-multiple-requests

    In your implementation, you have set the continue on error = false, which should break the execution if there are error but I haven't personally tested it. Could you change it on continueonerror to true and cathc the respose and display it in the console window if there are errors (there is a method in sample to display error)?

    I am having a feeling that there are error which are not thrown and hence you have mismatch in the count.

    Hope this helps.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Introducing the 2026 Season 1 community Super Users

Congratulations to our 2026 Super Stars!

Meet the Microsoft Dynamics 365 Contact Center Champions

We are thrilled to have these Champions in our Community!

Congratulations to the April Top 10 Community Leaders

These are the community rock stars!

Leaderboard > 🔒一 Microsoft Dynamics CRM (Archived)

Last 30 days Overall leaderboard

Featured topics

Product updates

Dynamics 365 release plans