Customer experience | Sales, Customer Insights,...

Azure Data Lake Microsoft Athena TrickleFeedService Documentation

(0) Share

Report

Posted on by Simon Morris

Hi,

Does anyone know of any documentation which describes how to use the Microsoft Athena TrickleFeedService files in Azure data lake?

I have these files but no idea how to use them.

any help would be very much appreciated.

All responses (2)

Answers (0)

Simon Morris 15 on at

Like (0)

Report

RE: Azure Data Lake Microsoft Athena TrickleFeedService Documentation

Thanks Carolina,

I have seen these docs and I am currently using those csv files at the moment but you have to load the whole file each time, what I really want is the trickle feed files so I only need to load the most recent updates, I just can't seem to find any documentation which explains how to use them.

The files below are in a different place in a separate folder in the data lake than the initial sync and snapshot files.

The trickle feed suggests they only contain updates and the delta file suggests the same but the delta file looks like this when you edit it:

I have no idea what that means!!!!. Thanks Simon
Suggested answer

Carolina Mogas on at

Like (0)

Report

RE: Azure Data Lake Microsoft Athena TrickleFeedService Documentation

Hello Simon,

I believe the information you are looking for is explained in this article - https://docs.microsoft.com/en-us/powerapps/maker/common-data-service/export-to-data-lake#continuous-snapshot-updates. You can also see this blog article for further information: https://powerapps.microsoft.com/en-us/blog/exporting-cds-data-to-azure-data-lake-preview/.

I assume that those files are associated with a specific folder entity in Storage Explorer. When changes occur for that entity, they are pushed to the respective CSV files in Azure Data Lake, using trickle feed. When data is written to the lake it is partitioned in years, as you can see in the screenshot you provided. Once this data is exported, you can consume the data using one of the options available in Discover Hub in the PowerApps Maker Portal.

This is an example article on how you can use PowerBI to analyze CDS data in Azure Data Lake: https://docs.microsoft.com/en-us/powerapps/maker/common-data-service/export-to-data-lake-data-powerbi.

Let me know if this was what you were looking for.

Thank you,
Carolina