Hi everyone, how scalable is Customer Insights ? apologies for the open ended nature of the question, but i have a potential customer who wants to create 3 million profiles in CI, and have them loaded to Power BI dataset & dashboard.
Thanks in advance....
It was not. We used their Lake (same with Chipotle) instead of the managed lake and used ADL for a container to hold the data to ingest. At a very simplified level, it was Synapse/Misc Other sources to data lake, mounted CDM folder for ingestion. I personally haven't done anything that used DataVerse at that level but as far as scalability - on the storage side, I don't see that being where a problem might be. before Power Platform Data Flows, if you were moving a lot of data (especially if it needed any form of ETL), power query wasn't going to be robust enough to pull it in. PPDF by contrast can handle quite a bit as long as the underlying source has the compute and you can use a source that supports query folding. I'd let one on the MS team speak to how much can be supported in the Managed lake (as opposed to using your own to host CI) but I would guess it's quite large as well. If I was told tomorrow on either project that our # of customers or activities were going to 4x what they are today, the only thing I'd be thinking about is the time difference on system refreshes, no concern about the volume. Unfortunately I just haven't done a large one that uses DataVerse but I think that's going to be pretty robust since MS manages it.
Hi Bill, many thanks for your useful insights.
Was this level of scalability (9m customers) leveraged using dataverse - i'd be very interested to hear of the high level architecture, and especially if dataverse was an integral part the platform you delivered.
Thanks for your useful feedback Ted (this is rtdh on new account). Can i ask if the data in the Chipotle project was integrated via Dataverse ? or through standard "Import Data" option in data sources menu in CI?
Ted mentioned Chipotle but I'd just add, with another big customer we have 9 million customer records, and with activities we have about 100 million Activities. In terms of Power BI, at 3 million profiles you shouldn't have any issues. When things get a little larger the model can get pretty bloated but that's more of a Power BI thing. The short answer though is at the scale you're looking at, it could easily triple or even more without even remotely causing a hiccup.
Great question! Customer Insights would scale to that with no problem. Please check out the Chipotle customer story where they have millions of customers and linked a billion activities to them: customers.microsoft.com/.../849868-chipotle-retailers-dynamics-365
--
Ted Way, PhD | PM Lead | Microsoft D365 Insights Apps AI
André Arnaud de Cal...
291,965
Super User 2025 Season 1
Martin Dráb
230,836
Most Valuable Professional
nmaenpaa
101,156