LAST UPDATE: RESOLVED
WORLDWIDE issue
Title: Users may have been unable to sign into your environment
User Impact: Users may have been unable to sign into your environment.
More info: Users may have received the following error when attempting to sign in: “Your account has been authenticated with Azure Active Directory, but there is no valid license assigned to your account in Business Central. Contact your system administrator to get a valid license.”
Final Status: We determined that the recent service update contained a code misconfiguration, which resulted in high resource utilization within the impacted portion of the infrastructure. We performed a rollback of the misconfigured code to the previous healthy configuration. We have confirmed via telemetry analysis and testing that users access to their environment has been restored.
Incident Start Time: Thursday, November 18, 2021, at 12:48 PM UTC
Incident End Time: Thursday, November 18, 2021, 2021, at 11:08 PM UTC
Preliminary Root Cause: A recent service update contained a code misconfiguration which resulted in impact.
Next Steps: We're reviewing our update procedures to better identify similar issues during our development and testing cycles.
Total Downtime: 10h 20m
Update 6 - 11:12 PM CET
Operations is continuing to monitor service health as the recovery progresses.
Update 5 - 7:22 PM CET
Partner should be able to see outage enlisted in Partner Center https://portal.office.com/Adminportal/Home#/servicehealth
Update 4 - 6:00 PM CET
After operations investigation, it has been currently identified high resource utilization in a portion of the infrastructure responsible for environment authentication as a result of a recent service update, resulting in impact. Operation telemetry analysis has determined that service health recovery is ongoing, and operations is investigating way to expedite its recovery.
In shorts, all clusters worldwide are slowly recovering and if not already done, it might take some time to have this back up and running.
It has been noticed tenant restart from few minutes up to few hours. Everything depends upon the load on every single cluster and it is not so linear, unfortunately.
Operations suggested a policy of wait and observe. It might be feasible that this might be potentially resolved in the next hours.
Update 3 - 4:50 PM CET
Operations seems to correlate the root cause internally and seems quite close to find out the root cause.
Still it is not confirmed yet.
Update 2 - 4:24 PM CET
Outage spread all over the world and officially declared impacting worldwide.
Operations still investigating root cause how and why this is happening with all workforces.
Update 1 - 4:00 PM CET
Operations is still checking for root cause and trying to have customers up and running as soon as possible
