The concept:
When utilizing recursive workflows with AI Agents, it would be ideal to have control over the number of concurrent instances of a specific workflow that are executed. This could be implemented within the workflow settings as an integer value, referred to as Maximum Concurrent Executions.
My scenario:
In my situation, I am attempting to crawl our company website. I've developed a recursive crawl workflow that invokes itself when new URLs are discovered. This also occurs when inserting documents into a Vector Database from another workflow. The recursive workflow crawls URLs found on the current webpage by calling itself via a webhook. However, when this process involves AI Agents, it could potentially reach the API limits of AI models.
I believe adding this feature would be advantageous because:
It could help prevent API LIMIT issues for workflows that incorporate AI Nodes.