Managing Rate Limiting in Workflows with Custom Code Actions

Developer Support APIs & Integrations
1

Hey devs! :waving_hand:

I’m tackling a common scenario:

We’ve got a Workflow that triggers a custom code action which sends out an API request. Nothing fancy—except the external API has a rate limit of 60 requests per minute. :warning:

What I need is a way to throttle the workflow enrollments so we don’t exceed that limit.

:backhand_index_pointing_right: I know there’s an app out there that seems to handle this pretty well… but I want to build this myself. I’m not looking to rely on a 3rd-party solution—I’d prefer to understand and control the throttling mechanism directly.

My main questions:

  • How would you implement this yourself?
  • What are your go-to strategies for queueing or delaying workflow executions to stay within rate limits?
  • Are there tools, patterns, or best practices you’d recommend for custom throttling logic?

Looking for some real-world advice or creative approaches to solve this DIY-style. :raising_hands:

Appreciate any thoughts or experiences you’re willing to share!

P.S.: This app is somehow doing it? But how?

2

Hi @pascalkremp,

Thanks for reaching out to the Community!

I would like to invite some members of our community who may offer valuable insights.— hey @TDwebdev, @HubDoPete, @Humashankar, @SteveHTM Could you share your advice with @pascalkremp?

Thanks for taking a look!

Diana

3

Hi @pascalkremp

Similar topic discussed in the Community, addressed around the challenges of API rate limiting during initial workflow enrollment, particularly when utilizing custom code actions that make external API calls.

A user shared their experience of encountering API throttling issues when enrolling a large number of contacts simultaneously, resulting in failed API calls.

Kindly go over the strategies, including allowing errors to propagate and trigger retries, implementing randomized delays to distribute the API call load over time, and segmenting workflows to enroll contacts at different times, thereby avoiding rate limit constraints.

Solved: HubSpot Community - API Rate Limiting During Initial Workflow Enrollment - HubSpot Community

Hope this helps - Happy to help further!!
Thank you very much and have a great one!
Warm regards

4

@Humashankar thanks for the answer. I already found this. I thought their might be a better way.
@DianaGomez maybe you can tag one of the product managers of workflows or custom code actions?

5

Hey @pascalkremp, just wanted to let you know that the workarounds @Humashankar and @HubDoPete pointed out are the current options available.

For new feature suggestions, please consider posting on our Ideas Forum here. If you find a similar idea, give it an upvote and share your unique use case in the comments.
If your idea is not already there, feel free to create a new idea.

Thank you for being part of the discussion!

Diana

6

Hi @pascalkremp and thanks @DianaGomez for the tag

Pascal we faced a similar problem in various use cases where workflows and apps are limited by 3rd party rate limits. Here are the principles of our approach if you are interested in coding a similar solution:

  1. Add a middleware/proxy layer where all calls to the 3rd party API can be centrally throttled. As middleware, the workflow will call your API endpoint, which calls the API on behalf of that workflow instance, then returns the result.
  2. Within the middleware, implement token-bucket rate limiting. Set the depth of the bucket and the rate at which tokens are added until the bucket is full, for example, 1 token per second, to suit your 60 per minute use case. Your code to refill the bucket is only required when the bucket is not full.
  3. When your middleware receives a call from your workflow, it must first request one or more tokens for immediate use from your token bucket code. If token(s) are granted, the API call is made and the result returned to the original caller. If no tokens are available, have your middleware pause up to a set limit before returning a failure message to the calling code, ideally within HubSpot’s 20-second workflow action timeout.

Levers you can adjust when tuning your token bucket to the external API include:

  • Token Bucket Depth
  • The rate at which tokens are re-added to the bucket
  • Max number of tokens in a single token request (to avoid one caller hogging all tokens)

This approach is similar to the technology employed in internet routers for throttling demand from many users over a network connection of limited bandwidth.

This approach should allow all your workflow requests to be shaped to fit the maximum allowed rate from the 3rd party API.

You can add more advanced techniques to this base method, depending on your use case:

  1. Create a token bucket per 3rd part API key. Each key increases your available rate limit, e.g. two keys to 2 x 60/minute. Internet routers use this approach for load sharing traffic across multiple network links to the same destination. Add a another link or API key to add more bandwidth.
  2. Create multiple token request queues for requesters of different priority calling the same 3rd party API. High priority workflow actions make high-priority token requests. Normal workflows make normal priority token requests. Your middleware always serves priority requests before normal requests. Again this approach is used for priority network traffic, such as delay-sensitive voice packets being routed before lower priority web/email traffic.

We use this approach at HubDo in various forms, as middleware for workflows or within an app. It’s a very deterministic approach for managing demands from multiple discrete requesters to a single limited resource.

Interested to hear what others have done to solve rate limiting in a deterministic way.

best

Pete

7

All - my previous answers to cuh questions have all revolved around decomposing sets of Contacts (or whatever object are in question) into distinct subsets that can be incrementally delayed. Example - all contats with first name beginning with ‘A’ get delay of 1 min, ‘B’, 2 min, etc. Within the workdlows, API calls will each have backoff approaches for the 429 errors, but the delay cannot be more than 20s cumulatively, otherwise the custom code step will fail for other limits.

I’ve never found any way to ‘cheat’ around this rather fundamantal system limit.

Steve