web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Community site session details

Community site session details

Session Id :
Dynamics 365 Community / Forums / Customer experience | Sales, Customer Insights,... / Clone bulk product bet...
Customer experience | Sales, Customer Insights,...
Answered

Clone bulk product between Business Unit

(3) ShareShare
ReportReport
Posted on by CU05031329-0 10
Hi guys,
I having this issue with cloning massive amount of record. Our system has more than 500,000 records of product in each Business Unit.
Every time, we create a new Business Unit, normally i would have to export all the product from a model BU, make some modifies and import to create in the new BU. Most of the field is unchanged, however some fields need to be updated with new-prefix.
This process take a really long time to complete, I need to prepare the file, the import job would take 1-2 days. Also, after the import, some products still missing.
Since we have many new BUs coming up, I need to find a way to automate this process. A tool or any method that can automatically read all product from a model BU, allow me to modify the field and start cloning to the new BU (hoping faster than import job).
Any suggestion is appreciated. Thank you very much!
Categories:
Microsoft Dynamics CRM
I have the same question (0)
  • Suggested answer
    Holly Huffman Profile Picture
    Holly Huffman 6,538 Super User 2025 Season 2 on at
    Good morning, evening, or afternoon :) depending on your location! 
    Hope you are well today! 
     
    • if you have access to a developer, creating a custom plugin or script may be beneficial which could read from the model BU and apply the modifications to create records in the new BU
    • Power Automate will likely not perform as well because its throttling with external API calls
    • there are other third party tools like Click 2 Clone which are meant for optimizing processes like this (I'm not an expert on these tools) but wanted to provide it as an option as I saw it while researching 
    • It may be good to chat through this with your partner and let them guide you on best practices 
     
    Hope this helps some! 
  • Suggested answer
    Tom_Gioielli Profile Picture
    Tom_Gioielli 2,792 Super User 2025 Season 2 on at
    DataFlows!
     
    This sounds like a perfect use-case for Dataflows, which can connect to your Dataverse environment, transform the data using Power Query, and then create/update records back in your Dataverse tables.
     
     
    For some context into the speed, I have a client where we need to update Price List Items once per day, according to some configuration rules. The Dataflow is able to find, modify/create about 150,000 records in around 15-20 minutes. It's truly one of the fastest and most powerful ways to do large data updates/creation in Dataverse.
     
    If this is something that needs to be repeated, add in some parameters to your Power Query where all you have to do is add the new pre-fix for specified fields and you are good to go. I really think this would be a great option for you.
     
    If this answer helped, please consider marking as verified.
  • Verified answer
    Daivat Vartak (v-9davar) Profile Picture
    Daivat Vartak (v-9d... 7,835 Super User 2025 Season 2 on at
    Hello CU05031329-0,
     

    You're facing a classic challenge with large-scale data duplication in Dynamics 365: slow import/export processes and potential data inconsistencies. Let's explore some efficient ways to automate the cloning of your 500,000+ product records across Business Units.

    1. Dataverse Web API (Recommended for Performance):

    • How it Works:

      • Use the Dataverse Web API to programmatically read product records from your model Business Unit.
      • Modify the necessary fields (e.g., prefix updates).
      • Use the Web API to create new product records in the target Business Unit. 

    • Advantages:

      • Significantly faster than import/export.
      • Provides granular control over data manipulation.
      • Can be automated using scripts or applications. 

    • Considerations:

      • Requires development skills (e.g., JavaScript, C#, Python).
      • You'll need to handle authentication and authorization.
      • Good error handling is essential. 

    • Example (Conceptual):

      • Use a script to:

        • Retrieve product records from the model Business Unit using a FetchXML query.
        • Iterate through the retrieved records.
        • Modify the required fields.
        • Use the POST method of the Web API to create new product records in the target Business Unit.
        • When using the POST method, make sure to set the owner of the record to the new business unit.

          •  

      •  

    2. Power Automate (For Automation and Integration):

    • How it Works:

      • Create a Power Automate flow that triggers when a new Business Unit is created.
      • Use the "List rows" (Dataverse) action to retrieve product records from the model Business Unit.
      • Use "Compose" actions to modify the necessary fields.
      • Use the "Add a new row" (Dataverse) action to create new product records in the target Business Unit. 

    • Advantages:

      • Automates the process.
      • Provides a visual interface for building workflows.
      • Integrates seamlessly with Dynamics 365. 

    • Considerations:

      • May be slower than the Web API for very large datasets.
      • Requires Power Automate skills.
      • You will need to implement a loop to process all 500,000 records.
      • Power Automate has a limit of 100,000 records that can be returned in a list rows action. To bypass this, you will need to implement paging. 

    • Paging:

      • Paging is a method to retrieve large amounts of data in smaller chunks.
      • You will need to use the skip token that is returned from the list rows action, to get the next set of records.

        •  

      •  

    3. Azure Logic Apps (Similar to Power Automate, More Scalable):

    • How it Works:

      • Azure Logic Apps provides a more scalable and robust platform for automating workflows.
      • You can use the same logic as Power Automate, but with more advanced features and capabilities. 

    • Advantages:

      • Highly scalable and reliable.
      • Provides a wide range of connectors and actions.
      • Excellent for complex workflows. 

    • Considerations:

      • Requires Azure Logic Apps skills.
      • May be more complex to set up than Power Automate.

        •  

      •  

    4. SSIS (SQL Server Integration Services) (For On-Premises or Dataverse Direct Access):

    • How it Works:

      • If you have on-premises Dynamics 365 or direct access to the Dataverse database, you can use SSIS to perform data migration.
      • Create an SSIS package to read product records from the model Business Unit, modify them, and insert them into the target Business Unit. 

    • Advantages:

      • Powerful and efficient for data migration.
      • Provides a wide range of data transformation capabilities. 

    • Considerations:

      • Requires SSIS skills.
      • Only applicable to on-premises or Dataverse direct access environments. 


      •  

    Key Recommendations:

    • Dataverse Web API (Best Performance): If performance is critical, use the Dataverse Web API. This is the fastest and most efficient method.
    • Power Automate (Automation): If you need a more user-friendly approach and automation capabilities, use Power Automate. Be aware of the record limits, and implement paging.
    • Testing: Thoroughly test your automation process in a non-production environment before deploying it to production.
    • Error Handling: Implement robust error handling to catch and resolve any issues during the data cloning process.
    • Chunking: When working with such large datasets, it is always best to break up the work into smaller chunks.
    • Business Unit Ownership: Ensure that the ownership of the records are set to the new business unit.
     
    If my answer was helpful, please click Like, and if it solved your problem, please mark it as verified to help other community members find more. If you have further questions, please feel free to contact me.
     
    My response was crafted with AI assistance and tailored to provide detailed and actionable guidance for your Microsoft Dynamics 365 query.
     
    Regards,
    Daivat Vartak

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Responsible AI policies

As AI tools become more common, we’re introducing a Responsible AI Use…

Neeraj Kumar – Community Spotlight

We are honored to recognize Neeraj Kumar as our Community Spotlight honoree for…

Congratulations to the November Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Customer experience | Sales, Customer Insights, CRM

#1
Tom_Gioielli Profile Picture

Tom_Gioielli 70 Super User 2025 Season 2

#2
Gerardo Rentería García Profile Picture

Gerardo Rentería Ga... 33 Most Valuable Professional

#3
Daniyal Khaleel Profile Picture

Daniyal Khaleel 32 Most Valuable Professional

Last 30 days Overall leaderboard

Featured topics

Hide selecting Contact for potential customer when creating quote

Product updates

Dynamics 365 release plans