web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

No record found.

Community site session details

Community site session details

Session Id :
Microsoft Dynamics AX (Archived)

Exporting to csv gets slows after 1800 records.

(0) ShareShare
ReportReport
Posted on by 60

Hello,

I am exporting some records from axapta 2009 to csv file, but after exporting 1800 records it gets so slow.

Actually I need to export 65 000 records, but I never could export it. I tried 10 000 of records it took 6-7 hours. But the interesting part till to 1800th record it is exporting 3 minutes. 

Can anyone has any idea ?

*This post is locked for comments

I have the same question (0)
  • Suggested answer
    Vilmos Kintera Profile Picture
    46,149 on at

    First of all - how are you exporting the data?

    If it is custom code, make sure it is executing server-side. Also writing to the file straight away is much slower. Instead you could fill the values into a TextBuf, then dump it to a file instead of continuous writing.

    msdn.microsoft.com/.../textbuffer.tofile.aspx

    www.cnblogs.com/.../2659731.html

  • Ertugrul Balveren Profile Picture
    60 on at

       I am sending the logic of the code. I deleted some lines. it has 569 lines of code. And it is running on server side.

      str 64 Butce;

       int referenceBuy;

       Amount          referenceBuyAmount, monthlyPay;

       Description     urunTipi;

       CustAccount custAccount1;

       BankChequeTable     bankChequeTable;

       Container           companyList = DirUtility::getDataAreaList(false);

       int curTime;

       int myCounter;

       ;

       curTime = timeNow();

       info(int2str(curTime));

       print int2str(curTime);

       try

       {

           perm = new FileIoPermission(#ExampleFile, #ExampleOpenMode);

           perm.assert();

           textBuffer.appendText("name;");

            //there are 57 columns more.              

           textBuffer.appendText("\n");

           while select smmBusRelTable

           index hint PartyIdx

           {

               DirPersonPartyDetail = DirPersonPartyDetail::find(smmBusRelTable.PartyId);            

               Address = Address::findCRM(smmBusRelTable.PartyId);

               textBuffer.appendText(strfmt("%1;",smmBusRelTable.Name));

               textBuffer.appendText(strfmt("%1;",smmBusRelTable.Email));

               while select CPMAreaTable

                index hint CustIdx

                 where CPMAreaTable.CustAccount == custAccount1

               {

                   textBuffer.appendText(strfmt("%1,",CPMAreaTable.AreaNameId));

               }

               textBuffer.appendText(strfmt(";"));

               textBuffer.appendText(strfmt("%1;",this.dispProjects(smmBusRelTable)));

               textBuffer.appendText("\n");

               myCounter++;

               if (myCounter == 1800)

               {

                  // we have 10 records

                  break;

               }

               print myCounter;

           }

           Lines = textBuffer.numLines();

            info(int2str(curTime));

            print int2str(curTime);

           try

           {

               if (textBuffer.toFile(#ExampleFile))

                   info( strfmt("File Generated as %1.total insert %2 Lines",#ExampleFile,Lines));

           }

           catch ( Exception::Error )

           {

               error ("Generated file error.");

           }

           CodeAccessPermission::revertAssert();

       }

       catch (Exception::Deadlock)

       {

           retry;

       }

    }

  • Suggested answer
    Vilmos Kintera Profile Picture
    46,149 on at

    I do not have AX 2009 to verify, plus you have a lot of custom logic in that code, so pasting it is not too useful. Also you did not use the preformatted tag, so parentheses looks weird, I cannot see what belongs where. You should only save values from the TextBuffer once ALL rows have been saved in there.

    First thing I noticed that the code is bad, why do you pull back the full record buffer for the tables when you only use 1 or 2 fields? Change it and have a field list.

    Remove the Info and Print calls, those are causing RPC roundtrip between server and client and causes slowness.

    Did you measure with Debugger or Trace Parser that which part of your code is taking a long time? Table operations, running specific code, or the actual toFile() part in a single go?

  • Ertugrul Balveren Profile Picture
    60 on at

    You are right. Posting to code is not useful because i did not use explanations tags and while i was deleting the codes i did not pay attention the logic of the code. I just post it to be understandable general logic of code. I did not use trace parser but i used debugger. There is nothing till to 1800th record after that record it gets slow on the calculation blog.

    I will use your advice and test the code after that let you know.

  • Ertugrul Balveren Profile Picture
    60 on at

    I did the things you said. But still it gets slow after 1830th records. There should be something different.

  • Suggested answer
    Vilmos Kintera Profile Picture
    46,149 on at

    I am afraid I cannot make any other improvement suggestions without seeing the exact code, knowing what hardware you are using, and checking the debugger/trace.

    The general advice is to collect a trace from a Developer/Test environment, isolate which code line is problematic and take action on that. You can find details on how to use the Trace Parser tool here:

    blogs.msdn.microsoft.com/.../dynamics-ax-trace-parser-part-3

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Responsible AI policies

As AI tools become more common, we’re introducing a Responsible AI Use…

Neeraj Kumar – Community Spotlight

We are honored to recognize Neeraj Kumar as our Community Spotlight honoree for…

Leaderboard > 🔒一 Microsoft Dynamics AX (Archived)

#1
Martin Dráb Profile Picture

Martin Dráb 4 Most Valuable Professional

#1
Priya_K Profile Picture

Priya_K 4

#3
MyDynamicsNAV Profile Picture

MyDynamicsNAV 2

Last 30 days Overall leaderboard

Featured topics

Product updates

Dynamics 365 release plans