Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Forums / Microsoft Dynamics NAV (Archived) / Uploading LARGE amount...
Microsoft Dynamics NAV (Archived)

Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

(0) ShareShare
ReportReport
Posted on by Community Member Microsoft Employee

Hi All,

REF: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

I need to upload to NAV 2018 around 0.9 to 1.1 MILLION Line Entries to the General Journal (Line Items) 

I have initially tried to use a provided Web Services URL and passed in all the parameters and it all "technically works fine" but I need MORE SPEED and MORE Reliability and Bigger or a Single Batch to upload 

Please can someone advise me on the BEST PRACTICES on how I should be uploading ?

(I know NAV is a bit of a baby product compared to SAP but I am hoping NAV has some cool solution and methodology and approach which I can use to upload a LARGE amount of data)

Initial findings is that RapidStart is Slow / possibly flaky / unreliable - (and as a Microsoft lad I am most disappointed) 

I have used Web Services it works but looking at batch sizes I will not perform this 500 times at around 2000 lines per time as this is prone to errors and boredom!

NAV now must be of a proper big boy product so taking SAP in mind how can I create a single one off task that will run in the background and perform all the usual and text book validations needed etc ?

Assuming that NAV has nearly 200,000+ installations then there must be a simple answer and solution ???

Help and pointers in the right direction are much appreciated 

Regards

Mark Dicken

ERP (SAP & NAV 2018 Dynamics) Data Migration Consultant 

UK Mob +447786530377

email@MarkDicken.com

*This post is locked for comments

  • Inge M. Bruvik Profile Picture
    Inge M. Bruvik 32,748 Super User 2024 Season 1 on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    Do you already have them in a SQL table in the same database as your NAV data? If so you can just write a simple codeunit to use the data to insert the G/J lines. If they are not in the same SQL database as your NAV data you can create a linked table. That means that NAV can read the data directly from the SQL table they are in and you don't have to import them into your NAV database.

    Please have a look at this link with information about using linked objects in NAV docs.microsoft.com/.../using-linked-objects

  • Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    Thankyou NorthW for your reply ...

    I have the Data in the correct format ... in both SQL Table or MS Access Table ... and yes I simply want to upload them as quick as possible ... if there is a Standard Upload Program then great I will happily use it if it properly posts to the API and does the usual validations that would be done with a normal online user then perfect  .... please point me in the direction of any standard upload program for my 1 million "General Journals" or please advise me of how many NAV 2018 Developer Hours I would expect to pay for to create a Upload Routine which is missing from NAV 2018 ...

    Many Thanks in advance for answers & suggestions ...

    Regards

    Mark

  • Suggested answer
    Inge M. Bruvik Profile Picture
    Inge M. Bruvik 32,748 Super User 2024 Season 1 on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    With this amount of data the best approach will be to read them into a temp table first with out any validation and then write some code to create the GJ lines based on the data in the temp table. Then you will have the fastest import and you have good control of the way you create the G/J lines.

  • Shaun Godber Profile Picture
    Shaun Godber on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    Mark,

    Break up into numerous smaller batches if possible (normally logical chunks of data are possible Period n in year y). This helps inserting and eventually posting

    Then write an XMLPort to import the data in these smaller chunks, it will take time but is quicker than RapidStart.

    The SQL direct insert route is not great as you need to make sure all of the data is correct (which if you don't know NAV (not saying you don't :) ) can be challenging)

  • Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    Unfortunately (for General Journals) it looks like I can only in a reliable way upload around 2000 records at a time ... I had technical issues with "just" 25000 rows : - ( ... I need to eventually upload around 1 million General Journal Lines and its a pity that NAV 2018 seems to not be able to cope with too many records when using Web Services ... if anyone else has pointers or suggestions then great else I may have to create and manually run 500 batches of 2000 lines (or perhaps 250 batches of 4000 lines)

    Having previously worked on many Data Migration Projects I generally want to automate as much as possible but with this Data Object (General Journals) its simply causing me pain because I need to upload with full validation and writing direct to SQL tables is not an option

    I wish there was some sort of (SAP) LSMW similar utility for NAV 2018 ??? that quietly works reliable in the background ...

    Regards

    Mark

  • Stefano Demiliani Profile Picture
    Stefano Demiliani 37,162 Most Valuable Professional on at
    RE: Uploading LARGE amount of data for just 1 Data Object (General Journal - Line Items)

    If your data are ok (so no need to trigger validations) you can direct writing on SQL. Using SqlBulkCopy or BULK INSERT for example  is very performant. We often use this method for very large data transfer but the data must be complete.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

December Spotlight Star - Muhammad Affan

Congratulations to a top community star!

Top 10 leaders for November!

Congratulations to our November super stars!

Tips for Writing Effective Suggested Answers

Best practices for providing successful forum answers ✍️

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,280 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,235 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans