Skip to main content

Notifications

Customer experience | Sales, Customer Insights,...
Answered

Data Transformation in CI for Parquet formatted files

(0) ShareShare
ReportReport
Posted on by 27

Hi , 

We have ingested Parquet formatted files into Customer Insights using the CI recommended Incremental refresh folder structure. All this works fine. Unfortunately we are unable to do any transformations within the Dynamics 365 Customer Insights upon this data. Is this normal, I would have expected to be able to do some transformation and data type conversions, what am I missing ? 

Any help greatly appreciated!

Many thanks, 

Mahesh

  • Verified answer
    Bill Ryan Profile Picture
    Bill Ryan 8 on at
    RE: Data Transformation in CI for Parquet formatted files

    Or Power Query - SQL has a pretty rich set of transforms but tbh, I'd probably opt for PQ in general over SQL.  They both get you there though.

  • MT_IGD Profile Picture
    MT_IGD 27 on at
    RE: Data Transformation in CI for Parquet formatted files

    Hi Bill,

    Thanks for quick reply! It seems that, for transformation, we need to take synapse route then.

    Thanks,

    Mahesh

  • Suggested answer
    Bill Ryan Profile Picture
    Bill Ryan 8 on at
    RE: Data Transformation in CI for Parquet formatted files

    Mahesh, if you're doing the CDM mount, the transformations will need to be done in advance.  Power Query is the only mechanism right now that'll let you do transformations on ingestion.  One other possible way around it is to setup a synapse serverless pool, create views on those parquet files and you can do several of the transforms in the view definition, but if you're already this far down the path it's probably not an option.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Announcing Our 2025 Season 1 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Vahid Ghafarpour – Community Spotlight

We are excited to recognize Vahid Ghafarpour as our February 2025 Community…

Congratulations to the January Top 10 leaders!

Check out the January community rock stars...

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,996 Super User 2025 Season 1

#2
Martin Dráb Profile Picture

Martin Dráb 230,853 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans