Skip to main content

Notifications

Announcements

No record found.

Dynamics 365 general forum
Answered

Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

Posted on by 75

One of the ADF pipe lines failed and the error message is "Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution."

The pipe line when run in the debug mode runs but when run through trigger , it fails with this error.

  • Suggested answer
    Mythe Profile Picture
    Mythe 75 on at
    RE: Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

    Hi Steve,

     Thanks for the suggestion , I checked the concurrency value and it is set to 1 and pipeline is set to run every 15 minutes. One of the executions was running for a long time and the further triggers exceeded the 100 limit and causing the error.

    After cancelling the long running process , the queue freed up. Also we might have to set the trigger to run every 30 minutes because the pipeline is taking longer time to complete.

  • Verified answer
    Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Azure Data Factory Pipe line run error - Pipeline execution throttled. If the concurrency value is set for a pipeline, a maximum of 100 runs will be queued for execution.

    Hi Mythe,

    According to the error message, you set the concurrency value, right? For example, you set concurrency value to 1and your pipeline has to run for 10 minutes . Then you set your pipeline triggered every 5 minutes, the second task would be added to queue until your first task is completed(10 minutes). If the number of task in the queue is over 100, you would get the error. Please try to reduce the pipeline run time or add the pipeline triggered time to have a try.

    Reference:

    Azure subscription limits and quotas - Azure Resource Manager | Microsoft Docs

    Data factory Run Fails - Microsoft Q&A

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Anton Venter – Community Spotlight

Kudos to our October Community Star of the month!

Announcing Our 2024 Season 2 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Dynamics 365 Community Newsletter - September 2024

Check out the latest community news

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 290,558 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 228,645 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,148

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans