Skip to main content

Notifications

Microsoft Dynamics 365 | Integration, Dataverse...
Answered

Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

(0) ShareShare
ReportReport
Posted on by 77

One of the ADF pipe lines failed and the error message is "Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution."

The pipe line when run in the debug mode runs but when run through trigger , it fails with this error.

  • Suggested answer
    Mythe Profile Picture
    Mythe 77 on at
    RE: Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

    Hi Steve,

     Thanks for the suggestion , I checked the concurrency value and it is set to 1 and pipeline is set to run every 15 minutes. One of the executions was running for a long time and the further triggers exceeded the 100 limit and causing the error.

    After cancelling the long running process , the queue freed up. Also we might have to set the trigger to run every 30 minutes because the pipeline is taking longer time to complete.

  • Verified answer
    Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

    Hi Mythe,

    According to the error message, you set the concurrency value, right? For example, you set concurrency value to 1and your pipeline has to run for 10 minutes . Then you set your pipeline triggered every 5 minutes, the second task would be added to queue until your first task is completed(10 minutes). If the number of task in the queue is over 100, you would get the error. Please try to reduce the pipeline run time or add the pipeline triggered time to have a try.

    Reference:

    Azure subscription limits and quotas - Azure Resource Manager | Microsoft Docs

    Data factory Run Fails - Microsoft Q&A

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

News and Announcements

Announcing Category Subscriptions!

Quick Links

December Spotlight Star - Muhammad Affan

Congratulations to a top community star!

Top 10 leaders for November!

Congratulations to our November super stars!

Tips for Writing Effective Verified Answers

Best practices for providing successful forum answers ✍️

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,359 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,370 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Product updates

Dynamics 365 release plans