API Throttling - Am I being throttled by Service API Limits?

Hello , so you are interested in knowing more about throttling in the context of Dynamics CE ? 

Well, then you have come to the right place, this community post aims to provide extra insights on this topic.

Index:

1. What is Throttling ?
2. How can I know if I am being throttled ?
3. Mitigations

What is Throttling ?

Throttling in Dynamics CE refers to a mechanism used by the system to control the amount of resources an application can consume within a specific time frame. As a result, when the server is under stress (e.g. extraordinary amount of requests, high resource consumption) instead of a response message indicating the request as successful, the client application can receive a message indicating the request has been throttled (limited for a certain period).

When it comes to  (Service protection API limits) these limits are enforced by user and webserver and apply to Web API and SDK Requests. Each Dynamics CE organization (e.g.: org.crm.dynamics.com) is associated with a certain number of web servers.

At the time of writing, Service protection API limits throttling can be enforced in 3 different ways according to the documentation below: 

Service protection API limits (Microsoft Dataverse) - Power Apps | Microsoft Learn 

Each surpassed limit is associated with a specific error code: 

Web service error codes (Microsoft Dataverse) - Power Apps | Microsoft Learn 

How can I know if I am being throttled ? 

 If you are throttled by Service API Limits the HTTP request status code of a Web API request will be 429. 

? Considering the following Python code: 

response = requests.get(f"{dynamics_endpoint}/accounts", headers={
        "Authorization": f"Bearer {access_token}",
        "Content-Type": "application/json"
    })

if response.status_code == 429:
    print(f"Error: {response.text}")

If many requests like the one above are fired within a small time frame, you can reach the Too many requests limit.  

Below follows the throttling error message for Too many requests for a Python custom console app that is making requests to the Web API:

Throttling can also affect other Dynamics CE integrations.

Below follows one example of the Execution Time limit affecting a copy activity in Azure Data Factory (ADF): 

If your Dynamics organization is connected to an Azure Application Insights instance you will also be able to see throttled requests (429s) in the resultCode column of the requests table: 

You can use the Web API query below to identify the user who made the request since that user_Id matches the azureactivedirectoryobjectid: 

org_url/api/data/v9.2/systemusers?$filter=azureactivedirectoryobjectid eq user_Id

Concerning SDK requests, when it comes to Microsoft.CrmSdk.XrmTooling.CoreAssembly nuget package, according to its release notes, it already has throttling support so you might not see the 429 HTTP Status Code since this package already handles it. Nonetheless SDK requests can also be throttled. 

https://www.nuget.org/packages/Microsoft.CrmSdk.XrmTooling.CoreAssembly/#release-body-tab 

Mitigations: 

• Using Multiple Application Users (How to Create Application user) in your App (since limits are enforced by user and webserver)

One of the common concerns with this approach is that Admins like to maintain the same user in Audit logs for better governance.

If you are using ServiceClient in C# to establish the connection to Dynamics CE you can set the CallerId using a different systemuserid so that this user will appear in Audit History logs instead:

serviceClient.CallerId = new Guid(systemuserid)

If you are using Python you can also leverage user impersonation by assigning the azureactivedirectoryobjectid of the user you want to impersonate to the CallerObjectId in the session.headers.

msdyn_iotalert_data = {
    "msdyn_description": "Description for Community Post"
}

session = requests.Session()
session.headers.update({
    "Authorization": f"Bearer {access_token}",
    "Content-Type": "application/json",
    "CallerObjectId": ""
})

# Send a POST request to the msdyn_iotalerts endpoint with the msdyn_iotalerts data in the request body
response = session.post(f"{dynamics_endpoint}/msdyn_iotalerts", json=msdyn_iotalert_data)

• Implement a Re-try mechanism

A re-try mechanism can work as a solution for the API throttling problem in Dynamics CE by allowing the application to automatically retry requests that have been throttled due to exceeding the API rate limits, thus reducing the likelihood of errors and providing a smoother user experience. You can review the code below, taken from this Microsoft document for inspiration:

/// 

/// Specifies the Retry policies
/// 
/// Configuration data for the service
/// 
static IAsyncPolicy GetRetryPolicy(Config config)
{
    return HttpPolicyExtensions
      .HandleTransientHttpError()
      .OrResult(httpResponseMessage => httpResponseMessage.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
      .WaitAndRetryAsync(
         retryCount: config.MaxRetries,
         sleepDurationProvider: (count, response, context) =>
         {
            int seconds;
            HttpResponseHeaders headers = response.Result.Headers;

            if (headers.Contains("Retry-After"))
            {
               seconds = int.Parse(headers.GetValues("Retry-After").FirstOrDefault());
            }
            else
            {
               seconds = (int)Math.Pow(2, count);
            }
            return TimeSpan.FromSeconds(seconds);
         },
         onRetryAsync: (_, _, _, _) => { return Task.CompletedTask; }
      );
}

• Using Batch Request

If you are facing errors related with the limits on the number of requests a good alternative is to use a Batch request and include your requests inside it rather than executing them individually. This also leads to reduced network traffic since each request requires its own network round trip. 

On the other hand, large sized batches increase the chance you will encounter execution time limits, so it is better to start with small batch sizes until you find the best balance. 

Feel free to check the code below for inspiration: 

//not using a batch request
for (int i = 0; i < numberOfRequests; i  )
{
    Entity invoice = new Entity("invoice");
    serviceClient.Create(invoice);
}
//using Batch Request
ExecuteMultipleRequest batchRequest = new ExecuteMultipleRequest();
for (int i = 0; i < numberOfRequests; i  )
{
    Entity invoice = new Entity("invoice");
    var createRequest = new CreateRequest() { 
    Target = invoice
    };
    batchRequest.Requests.Add(createRequest);
}
ExecuteMultipleResponse batchResponse = (ExecuteMultipleResponse)serviceClient.Execute(batchRequest);

• Leveraging Parallelism

If you read until this point, you probably understand that handling throttling is a game of finding the best balance with the resources available. Parallelism plays under the same rules - If increased to an extreme it can be the cause of throttling, but when leveraged wisely, it can do wonders for data throughput.

//For Loop without Parallelism
for (int i = 0; i < numberOfRequests; i  )
{
 Entity invoice = new Entity("invoice");
 serviceClient.Create(invoice);
}
//For Loop using Parallelism
Parallel.For(0, 10, new ParallelOptions { MaxDegreeOfParallelism = 4 }, i => 
{
 Entity invoice = new Entity("invoice");
 serviceClient.Create(invoice);
});

Also when using .NET and sending requests in parallel take notice you can apply configuration changes like the ones mentioned on this doc so that the requests are not limited by default settings:

Hope this read provided you some new insights about Throttling 

Thank you for reading.