Dynamics Community Forum Thread Details

Ingesting data from BC and Dataverse to Fabric

Hello, I am trying to find out the best architecture to ingest data from both BC and Dataverse into Fabric. Since we don’t have much experience with Python and we don’t have many transformations to perform, I am trying to avoid using notebooks.

Currently, I am considering two options:

Ingesting data using Dataflow Gen2 – The issue here is that we need to manage incremental refresh, especially when records get deleted from one of the sources (either BC or Dataverse).
Using the BC2ADLS tool and Link to Azure Synapse– This would ingest data into Azure Data Lake, and finally ingest data into Fabric using shortcuts (if possible).

Which of the two approaches is better in terms of cost and performance, and are there other approaches to consider?

Categories:

Power Platform integration

Dynamics 365 general

Hi,

You're comparing two approaches for ingesting data from Business Central (BC) and Dataverse into Microsoft Fabric, and you're looking for guidance on cost, performance, and alternatives.

✅ Summary of the Two Approaches

Option 1: Dataflows Gen2

Pros:
- No-code/low-code setup.
- Native integration with Fabric Lakehouse.
- Ideal for simple transformations and direct Power BI consumption.

Cons:
- Limited control over incremental refresh, especially for deletions.
- May require manual logic or workaround for soft deletes or record tracking.
- Performance can degrade with large datasets or complex joins.

Option 2: BC2ADLS + Azure Synapse Link + Fabric Shortcuts

Pros:
- More scalable and performant for large datasets.
- Better suited for incremental loads, including deletes, via CDC or snapshot logic.
- Fabric Shortcuts allow zero-copy access to data in ADLS.

Cons:
- Requires more setup and Azure resources (e.g., Synapse, ADLS).
- Slightly higher cost due to multiple services involved.
- May need basic scripting or orchestration (e.g., Data Factory or Synapse pipelines).

💡 Recommendation

If your team prefers low-code and has modest data volumes, Dataflows Gen2 is simpler and cost-effective. However, if you're dealing with larger datasets, need robust incremental refresh, or plan to scale, the BC2ADLS + Synapse + Fabric Shortcuts approach is more future-proof.

🔍 Alternative Approaches to Consider

Power Platform Dataflows + Data Factory

Use Power Platform Dataflows to ingest from Dataverse and BC, then orchestrate with Azure Data Factory for incremental logic and push to Fabric.

Dataverse Export to Data Lake (CDM format)

Use the Export to Data Lake feature for Dataverse, then link to Fabric via shortcuts.

Direct API Integration with Azure Functions or Logic Apps

For lightweight ingestion, use APIs to pull data into ADLS or Fabric via custom connectors.