Hi all,
we are building a Customer Insights system with customer data mainly coming from Dataverse. We face several issues when using the import method "Dataverse".
Has anybody observed similar issues or is there something wrong with our CI instance? Are these bugs or considered "normal" behaviour?
1. Dataverse data can not be filtered or transformed in any way and is always loaded "full table" with all columns
This leads to loads of data in CI that are not needed and just swamp th system. Even worse, it makes the unification process very difficult. E.g. how can I ignore/filter deactivated D365 contacts during M3?).
What is the recommended best practice to transforem Dataverse data? The PowerQuery Dataverse connector is gone, should we use PowerQuery oData instead (I hope not)?
2. Dataverse "connector" cannot cope with linebreaks
It creates many entries in the contact_Corrupt table. These are most probably caused by linebreaks in text fiels that are not properly handeled by CSV conversion when data travels through the internal MS data lakes.
How can these be fixed? Do we need to ask the data owner to clean up their linebreaks in Dynacics 365? They most probably will not care because it "works for them".
Again this would not be a big issue if there was a practical way to transform Dataverse data before ingesting into CI (see 1).
3. Several (custom) Dataverse tables missing from selection
Several (but not all) custom objects/tables from Dynamics 365 are not displayed for selection. However, these tables are visible and accessible though other connectors (e.g. Power Query from Power BI) - but not in Customer Insight. So we can not use this data at all (!?!)
4. Cannot remove Dateverse tables from selection after initial load
Once a Dataverse table has been selected and is loaded, it can not be removed/deleted anymore (selection checkbox is greyed out) - even if no further processing of the table has ever been configured (M3, Relationships etc.). Is this a bug or a feature?
4. Dataverse refresh/update duration
Updates fo Dataverse data can take very long, especially adding new tables within the connection can take hours.
