web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Community site session details

Community site session details

Session Id :
Dynamics 365 Community / Forums / Finance | Project Operations, Human Resources, ... / Apache Kafka Integrati...
Finance | Project Operations, Human Resources, ...
Unanswered

Apache Kafka Integration with D365 FO using Logic Apps

(0) ShareShare
ReportReport
Posted on by Usama Tur Razzam 115

Hello Integration Champs,
We are integrating our POS sales with D365 FO. POS sales are syched at apache Kafka and we need to fetch sales data through Apache Kafka Rest proxy using logic app.

Which means is we will be using Logic Apps as a bridge between Kafka and ERP. Kafka returns data to logic app in form of json.

We are confused how we should pass the data to ERP? Either using custom web service where we pass same json to ERP? or we go for Composite entity approach in which case we would have to convert json to XML and then add needed tags in that for composite entity and then pass this xml using Batch Data API.

All the suggestions are welcome. Please keep in mind we have around 60k orders per day with 20-40 lines each order.

I have the same question (0)
  • Martin Dráb Profile Picture
    Martin Dráb 237,990 Most Valuable Professional on at

    Whether you should use a data entity or a contract designed by yourself depends a lot on what data it is and what you want to do with it in F&O.

    If you're talking about sales orders, using a data entity makes a good sense to me. Using a custom service would mean manually designing contracts with all fields you need and either duplicating all the logic or actually using the data entities under the hood.

    Another benefit of using data entities is that you can utilize data management APIs to import data in bulk, which may be important in your case.

  • Usama Tur Razzam Profile Picture
    Usama Tur Razzam 115 on at

    Hi Martin,

    Your reply is much appreciated always.

    For each call to kafka we get list of orders which needs to go to different legal entities. Do you recommend concating them according to each legal entity and then converting them in xml and add missing tags for composite entities and then pass them on to respective legal entitites?

    For your point about using sales orders entity, we can't as we have a complex BL to consolidate them based on factors and then posting BOMs and posting sales. So the data will go to our staging tables.

    What would you suggest now? Batch Data API or a custom service?

  • Martin Dráb Profile Picture
    Martin Dráb 237,990 Most Valuable Professional on at

    I'm not 100% sure what you mean by " bout using sales orders entity, we can't as we have a complex BL [...]. So the data will go to our staging tables", but your explanation doesn't sound to me like anything that would prevent from using data entities. Correct me if I'm wrong.

    I cant tell you what exactly you should do, e.g. whether you should use composite entities. I don't know enough about your situation. But if you're worried about performance (which seemed to be the case), I would strongly consider data management APIs.

    By the way, your choice isn't just "one of DM APIs" vs. custom services. You could also use data entities through OData.

  • Usama Tur Razzam Profile Picture
    Usama Tur Razzam 115 on at

    Sorry for my ambiguous statement. I will try to rewrite again.

    As a business process we won't be creating one ERP Sales Order for one sales Order in our POS. This is the reason we wont be using sales order entity (this is what i meant). We have certain parameters of consolidation which will come in play by our batch job which will run on our sales data (received from logic app) residing in our staging tables.

    Our POS data is complex having multiple objects and multiple list of objects within one sales order so using OData wont be a good choice IMHO as for each object in header and objects in lists we will be separately calling simple data entities using Odata to store the data.

    That's why we are only left with 2 options either a composite entity or a custom service.

  • Martin Dráb Profile Picture
    Martin Dráb 237,990 Most Valuable Professional on at

    You don't have to use standard entities. If you want to import data to your custom tables, you can create data entities for them.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Responsible AI policies

As AI tools become more common, we’re introducing a Responsible AI Use…

Neeraj Kumar – Community Spotlight

We are honored to recognize Neeraj Kumar as our Community Spotlight honoree for…

Congratulations to the November Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Finance | Project Operations, Human Resources, AX, GP, SL

#1
Abhilash Warrier Profile Picture

Abhilash Warrier 669 Super User 2025 Season 2

#2
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 449 Super User 2025 Season 2

#3
Martin Dráb Profile Picture

Martin Dráb 384 Most Valuable Professional

Last 30 days Overall leaderboard

Featured topics

End-of-life Announcement for Microsoft Dynamics GP

Product updates

Dynamics 365 release plans