Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Forums / Small and medium business | Business Central, N... / Webhook goes to sleep ...
Small and medium business | Business Central, N...
Answered

Webhook goes to sleep after no user sessions

(0) ShareShare
ReportReport
Posted on by T Henry 74

I have an interesting issue with BC and webhook subscriptions.  We have found that it almost looked like the webhooks were on a schedule, they would stop at 7pm and start up again in the morning around 8am.  We looked at telemetry from Azure Application Insights (image below) for our web service and noticed that pattern.

What we have now concluded is that it seems when user sessions are gone there are no webhook events being fired.  Once users start logging in to BC in the morning the queue of events will trigger all at once (those are the spikes in the graph below).

I cannot find any information about BC going idle when there are no users for webhooks.  Does anyone have any insight?

pastedimage1647631649524v1.png

  • ajkauffmann Profile Picture
    ajkauffmann 117 on at
    RE: Webhook goes to sleep after no user sessions

    I totally agree, it is a workaround while we wait for a permanent fix.

  • RonKoppelaar Profile Picture
    RonKoppelaar 5 on at
    RE: Webhook goes to sleep after no user sessions

    AJ although this will work, this looks like a workaround to me. Notifications are send by the platform, and it shouldn't matter who triggered the update. External systems are relying on these notifications. S2S is the modern way of integrating 3rd party systems. Basic Auth has been deprecated. This really needs to be solved by MS before 1st of October...

  • Verified answer
    ajkauffmann Profile Picture
    ajkauffmann 117 on at
    RE: Webhook goes to sleep after no user sessions

    The reason that application accounts can't schedule webhooks has never been clearly explained by Microsoft. My guess is that it would open a wide hole in the licensing mechanism because the application account is meant for external users (not requiring a license) who should not start internal processes like posting routines. Another reason could be that the application account does not represent a real user, and as a result the audit field InsertedBy and ModifiedBy would not tell who created certain entries. Auditors will not like that.

    Anyway, an unexpected result is that triggering the webhook notification does not work for API requests done with service to service authentication.

    A workaround is to permanently schedule a recurring job for codeunit "API Webhook Send". Maybe in the same way as is done with job "Telemetry Management" which reappears even after it was deleted. This is managed by codeunit "Telemetry Management", procedure ScheduleDailyTelemetryAfterCompanyOpen.

    To avoid duplicate job queue entries (for API requests that do have permission to start the job queue) you should create an event subscriber to event OnCanCreateTask from codeunit "API Webhook Notification Mgt." and then set CanCreateTask to false and IsHandled to true.

    Microsoft is aware of this problem and they are working on a fix as we speak. But currently no estimated date when a fix will be available.

  • T Henry Profile Picture
    T Henry 74 on at
    RE: Webhook goes to sleep after no user sessions

    I think I found the answer to the issue but it's not as I expected.  Our API is using service-to-service authentication.  We have application account in Business Central and a user account for the application.   I believe the application is using delegated admin permissions.  Delegated admins do not have permissions to task schedule or job queues (as mentioned above by Microsoft support).  That is disappointing because it seems the only other way to trigger webhooks is to use basic authentication, that is being deprecated very soon so not an option.

    I thought then if I used an Azure Logic App to trigger record events instead of webhooks.  However it seems logic apps have the same delegated permission issues and will add the event to the queue but not trigger.  Only when I logged into Business Central and updated a record did it trigger my event but all other events in the queue generated from the API.

    In the end the only recourse is to use a recurring logic app that will check for new purchase orders every X minutes.  It will loop through those purchase orders and perform an event.  In our case we are sending a message to a service bus topic.

    I don't understand the reason why an application cannot work with webhooks. Should this work and something is incorrect with our configuration?

    References

    Microsoft Idea (dynamics.com)

    Configuring Business Central for Azure Active Directory authentication and OAuth (2) – Kauffmann @ Dynamics 365 Business Central ajkauffmann 

    Dynamics 365 Business Central, Service-to-Service authentication (OAuth) and Task Scheduler – Stefano Demiliani Stefano Demiliani 



  • T Henry Profile Picture
    T Henry 74 on at
    RE: Webhook goes to sleep after no user sessions

    We are in contact with someone from Microsoft.  This is what they are saying

    Webhooks are not triggered by user's login. The system works like this: 

    When a modification to a record is done, we check if there are any webhook subscriptions for this record. If yes, we insert the notification record and schedule a Job Queue to send these notifications in 30s. If there are any other records modified, we will postpone the Job Queue by 30s. 

    From the telemetry it looks like they are using a user that has no rights to schedule Job Queues. This could be Delegated Admin, Internal Admin or a delegated app authentication. This change will insert a notification, however it cannot schedule a job queue. Then if a user with rights to schedule Job queues logs in and any of the records is updated then they send the job queues. 

    The user account that we use for API entries has more permission sets than my account. The only difference between the accounts is that the API account Authentication Status is set to Disabled.

    pastedimage1651789000566v1.png

    What am I missing here? Is there a specific permission just for webhook events?

  • Suggested answer
    MahGah Profile Picture
    MahGah 15,433 on at
    RE: Webhook goes to sleep after no user sessions

    Based on Stefano feedback this is a strange behavior. I think you should ask your partner to open ticket with Microsoft.

  • T Henry Profile Picture
    T Henry 74 on at
    RE: Webhook goes to sleep after no user sessions

    It is still happening.  We left the messages in the servicebus queue and looked at them on Monday.  We had a user log in over the weekend to do some work.  You can see the messages are all grouped and sent at the same time.

    I'm at a loss here.  Webhooks should not be session/user activity driven.   Is this an issue I could raise directly with Microsoft support?

    pastedimage1648135082872v1.png

  • T Henry Profile Picture
    T Henry 74 on at
    RE: Webhook goes to sleep after no user sessions

    Actually I forgot that I set up our production environment to use Application Insights for telemetry.

    I'll do some research there to see if there are webhook events that are failing reaching the destination.

    Any guidance on the query to use would be greatly appreciated.

  • T Henry Profile Picture
    T Henry 74 on at
    RE: Webhook goes to sleep after no user sessions

    Hi Stefano Demiliani,

    It is indeed strange.  Our webhooks are hitting an Azure app so it should be reliable.

    Is there telemetry I can look at on the Business Central side to see any webhook events for errors?

  • Suggested answer
    Stefano Demiliani Profile Picture
    Stefano Demiliani 37,162 Most Valuable Professional on at
    RE: Webhook goes to sleep after no user sessions

    Webhooks are not linked to users logged in the system but to entity modifications. Webhooks subscriptions expires in 3 days (default value). They can be renewed by sending a PATCH request to the subscriptions API as described in my post above. The described behavior of this post sounds strange.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Congratulations 2024 Spotlight Honorees

Kudos to all of our 2024 community stars! 🎉

Meet the Top 10 leaders for December

Congratulations to our December super stars! 🥳

Start Your Super User Journey Pt 2

Join the ranks of our community heros! 🦹

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,861 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,540 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans