Webhook goes to sleep after no user sessions

(0) Share

Report

Posted on by T Henry

I have an interesting issue with BC and webhook subscriptions. We have found that it almost looked like the webhooks were on a schedule, they would stop at 7pm and start up again in the morning around 8am. We looked at telemetry from Azure Application Insights (image below) for our web service and noticed that pattern.

What we have now concluded is that it seems when user sessions are gone there are no webhook events being fired. Once users start logging in to BC in the morning the queue of events will trigger all at once (those are the spikes in the graph below).

I cannot find any information about BC going idle when there are no users for webhooks. Does anyone have any insight?

I have the same question (0)

All responses (13)

Answers (1)

Suggested answer

MahGah 15,529 on at

Like (0)

Report

Hi

I could be wrong here.

According to the following article "In short, a webhook is a push notification. The “don’t call us, we’ll call you” scenario of web services. You tell Business Central in what resource you are interested (e.g. Item, Customer, etc.), and they’ll call you in case there is any change. With resource I mean any of the existing APIs, including custom APIs."

Hence, my assumption is when no user use BC then no change and hence webhook has no change to send. Just for test can you login in BC after those hours and change something? Or schedule a job queue that change something?

www.kauffmann.nl/.../

Was this reply helpful? Yes No
T Henry 74 on at

Like (0)

Report

Hi MahGah,

We are using APIs to create sales orders and then automatically create Purchase Orders. We have a webhook for the purchase orders when the are created to send a push. So there are no users involved in that experience. So when these purchase orders are created after hours we are not seeing the events being fired. What is more interesting is if the purchase order is created on Friday evening or the weekend some of those purchase order webhook events do not fire at all. I don't know if it is an expiry issue.

We have a PATCH request that renews the subscription every midnight UTC. I don't know if that is related.

Was this reply helpful? Yes No
Suggested answer

MahGah 15,529 on at

Like (0)

Report

Hi

Thanks for info. Looks like webhook is active for 3 days ( demiliani.com/.../ ). I hope some one like Stefano Demiliani or Roberto Stefanetti or yzhums or JAngle or TeddyH can provide more info here.

Was this reply helpful? Yes No
Suggested answer

Stefano Demiliani 37,166 Most Valuable Professional on at

Like (0)

Report

Webhooks are not linked to users logged in the system but to entity modifications. Webhooks subscriptions expires in 3 days (default value). They can be renewed by sending a PATCH request to the subscriptions API as described in my post above. The described behavior of this post sounds strange.

Was this reply helpful? Yes No
T Henry 74 on at

Like (0)

Report

Hi Stefano Demiliani,

It is indeed strange. Our webhooks are hitting an Azure app so it should be reliable.

Is there telemetry I can look at on the Business Central side to see any webhook events for errors?

Was this reply helpful? Yes No
T Henry 74 on at

Like (0)

Report

Actually I forgot that I set up our production environment to use Application Insights for telemetry.

I'll do some research there to see if there are webhook events that are failing reaching the destination.

Any guidance on the query to use would be greatly appreciated.

Was this reply helpful? Yes No
T Henry 74 on at

Like (0)

Report

It is still happening. We left the messages in the servicebus queue and looked at them on Monday. We had a user log in over the weekend to do some work. You can see the messages are all grouped and sent at the same time.

I'm at a loss here. Webhooks should not be session/user activity driven. Is this an issue I could raise directly with Microsoft support?

Was this reply helpful? Yes No
Suggested answer

MahGah 15,529 on at

Like (0)

Report

Based on Stefano feedback this is a strange behavior. I think you should ask your partner to open ticket with Microsoft.

Was this reply helpful? Yes No
T Henry 74 on at

Like (0)

Report

We are in contact with someone from Microsoft. This is what they are saying

Webhooks are not triggered by user's login. The system works like this:

When a modification to a record is done, we check if there are any webhook subscriptions for this record. If yes, we insert the notification record and schedule a Job Queue to send these notifications in 30s. If there are any other records modified, we will postpone the Job Queue by 30s.

From the telemetry it looks like they are using a user that has no rights to schedule Job Queues. This could be Delegated Admin, Internal Admin or a delegated app authentication. This change will insert a notification, however it cannot schedule a job queue. Then if a user with rights to schedule Job queues logs in and any of the records is updated then they send the job queues.

The user account that we use for API entries has more permission sets than my account. The only difference between the accounts is that the API account Authentication Status is set to Disabled.

What am I missing here? Is there a specific permission just for webhook events?

Was this reply helpful? Yes No
T Henry 74 on at

Like (1)

Report

I think I found the answer to the issue but it's not as I expected. Our API is using service-to-service authentication. We have application account in Business Central and a user account for the application. I believe the application is using delegated admin permissions. Delegated admins do not have permissions to task schedule or job queues (as mentioned above by Microsoft support). That is disappointing because it seems the only other way to trigger webhooks is to use basic authentication, that is being deprecated very soon so not an option.

I thought then if I used an Azure Logic App to trigger record events instead of webhooks. However it seems logic apps have the same delegated permission issues and will add the event to the queue but not trigger. Only when I logged into Business Central and updated a record did it trigger my event but all other events in the queue generated from the API.

In the end the only recourse is to use a recurring logic app that will check for new purchase orders every X minutes. It will loop through those purchase orders and perform an event. In our case we are sending a message to a service bus topic.

I don't understand the reason why an application cannot work with webhooks. Should this work and something is incorrect with our configuration?

References

Microsoft Idea (dynamics.com)

Configuring Business Central for Azure Active Directory authentication and OAuth (2) – Kauffmann @ Dynamics 365 Business Central ajkauffmann

Dynamics 365 Business Central, Service-to-Service authentication (OAuth) and Task Scheduler – Stefano Demiliani Stefano Demiliani

Was this reply helpful? Yes No