How many contacts go through a workflow concurrently?

Data Hub Data Hub
1

I have created a workflow that uses custom javascript code. As part of that script 2 API calls are made to our internal endpoints and 2 are made to the Hubspot API. I have over 100k contacts that are to be initially enrolled in the workflow, (after that, new contacts only). Before doing this I want to ensure that both our internal API will not be flooded with requests and also our Hubspout API rate limites are not reached.

If I enrol 100k contacts, how many contacts are concurrently in the workflow at the same time running the script?

And if this is a large number do I have any way to throttle the amount of contacts concurrently running the javascript script?

2

Hi @JWilson7361,

Out of the box, there aren’t any features to throttle or limit workflow enrollment at the moment. There’s a similar thread here which briefly discusses some options including a timer and throwing an error which would cuase the workflow action to rerun: https://community.hubspot.com/t5/9881-Operations-Hub/API-Rate-Limiting-During-Initial-Workflow-Enrollment/td-p/420973

Hope this helps!

3

Yes, you will likely hit your rate limits for hubspot at least. Hubspot workflows might not trigger all 100,000 contacts at once but it will do more than the API rate limits allow. I have recently come across this issue and found a hacky resolution for it. I have around 25k contacts that run through my workflow each fortnight and this seems to work fine.

The other thread suggests using a JS timer, however, I found this didn’t work as the custom workflow actions time out after 20 seconds.

You can throw an error in your API call which I found is the best solution to ensure the workflow will continue to retry actions before moving on to the next step. This one is quite important. However I imagine even with large numbers like this going through this workflow it will have to retry so many times before success that, as some discussion on the hubspot forums suggest but documentation doesn’t say explicitly, it will stop after a certain point.

In order to trickle my objects through the workflow I did a bit of a hacky job utilising the lead rotation functionality, an ‘owner’ property that I use exclusively for CRM purposes and the out of the box delay function for workflows.

  • First, you would need to have an owner property for the object that you could use where it isn’t important to retain the existing information. So I wouldn’t recommend using ‘contact owner’ if this is used for anything else. You may want to create one.
  • Once you have this, you would use the hubspot ‘Rotate Record to Owner’ workflow action to rotate the leads between a number of different owners, this number should be determined based on the number of delays you want to put in place.
  • Then you would branch out using the value equals branching in HubSpot workflows for each of the owners you assigned the leads from the previous step.
  • Under each branch you can add a specific delay that you want to allow and this should increment in value as you move along the branches. ie, 10 minutes, 20 minutes, 30 minutes etc.

If you use this in combination with throwing errors in your API call, this will trickle your contacts through, ensure that if they hit any rate limit errors, they’ll be retried but it won’t feed them all in at once.

More information on the rate limits is available here: API usage guidelines and limits - HubSpot docs. Depends on the type of account you have but presuming you have at least professional for ops hub they are:

150 per 10 seconds (for each app)

500,000 per day (for your whole account)

And if your API calls are utilising the search endpoint:

  • The Search API endpoints are rate limited to four requests per second per authentication token

So you may want to work backwards from the rate limits to determine how much of a delay you need to add between each lead rotation group.

Hope that helps and let me know if you need any more context on the above.

4

I’d like to provide an updated method for this solution now that HubSpot has random distribution branches for Marketing Hub Pro & Ent. Rather than use an owner property and rotation action, you can create branches directly based on percent splits. If you have the required Hub and tier, this new method has the added benefit of being quicker to configure and is not coupled to user profiles that could be deleted.

5
  • In your custom JavaScript, you could add setTimeout or setInterval functions to ensure that the script doesn’t run all at once. For example, you could delay every API call by a certain number of milliseconds to reduce the speed at which requests are sent.
  • Rate Limiting Libraries/Approaches:
    You can utilize third-party libraries like Bottleneck or Rate-limiter-flexible to control the rate at which requests are sent. These libraries help throttle API requests based on specific limits you set.
  • Control Workflow Entry:
    You may also want to stagger the initial entry of the 100k contacts into the workflow. This way, you’re not overwhelming your API systems at once. For example, you could gradually enroll contacts in the workflow over a period of hours or days.
6

@JWilson7361, and everyone else finding this thread, we have another update from HubSpot in the form of a public beta that allows configurable rate limits on custom coded actions. This doesn’t help if you’re not using CCAs. But this has simplified my workflows considerably where I need to throttle my CCAs.