API Rate Limiting During Initial Workflow Enrollment

[bryce-corey]

Hey All,

I'm running into an issue, wondering if anyone has had the same experience, and could share if they found a solution!

When making API calls within the Custom Code Action in a workflow, I am running into an API throttling issue (“You have reached your secondly limit”) when a large number of contacts are enrolled at the same time in the workflow. This seems to be due to all of the contacts going through the custom code in their workflow at the same time, and because the API calls are being made using the same HAPIKEY, HubSpot is throttling the amount of calls being made.

For example, I enabled a workflow that enrolled 5,000 contacts automatically, and 4,000 custom code actions failed because of the rate-limiting. Luckily, I was able to remove the API call from that specific workflow by handling some logic outside of the custom code. However, the next plan we have is to format phone numbers on our contacts. Since we have to retrieve the phone number through an API call within the custom code and we will be enrolling around 10,000 contacts, we cannot avoid the problem this time around.

My initial thought was to set some sort of delay within the workflow, but I realized that they would all have the same delay and then run at the same time, which would defeat the purpose. The same goes for setting a timeout in the JS within the custom code. I’m a little lost at where to go from here and am afraid we might have to enroll them in different segments.

If you have any ideas of how to get around this, please let me know!

[Unmatched]

Hey! We've built an app that will be able to help you with this. It's called Time Turner, and it allows you to throttle your HubSpot workflows and drip-feed your emails out over time. 

Let me know what you think? 

Here's the install link for Time Turner!

[KimM]

Bit late to the game here but thought I'd add my insight on how I handled this. I used a combination of different approaches throttle the API calls.

Exactly what has been previously mentioned, it's important to throw the error in your catch statement. This just means that hubspot will continue to retry this action until it succeeds. Some people on the discussion boards suggest that the retries will only happen certain number of times, but the documentation doesn't clearly state it and I haven't come across it yet.

I also implemented a hacky solution using the 'Rotate Leads to Owner' workflow action, an owner property I use exclusively for this purpose, and the built-in delay function.

Check my response out here for more details: https://community.hubspot.com/t5/9881-Operations-Hub/How-many-contacts-go-through-a-workflow-concurr...

[spotthehub]

So when opening the workflow I have nothing in the retries section: 

[spotthehub · Accepted solution]

Hah @bryce-corey I had the exact same issue today and tried a JS timer as well.  Did you end up finding a solution?  Support suggested throwing an error instead of catching the error, suggesting that would cause the workflow job instance to rerun. 

[OTanti]

I createa a nodejs code snippet to create a random delay of between 0 and 9 seconds before the code snippet which sends API calls. In case this helps:

exports.main = async (event, callback) => {

  // Function to create a random delay between 0 seconds (0 ms) and 9 seoncds (9000 ms)
  const randomDelay = () => {
    const min = 0; // 0 milliseconds
    const max = 9000; // 9 seconds in milliseconds
    return Math.floor(Math.random() * (max - min + 1)) + min;
  };

  // Delay execution
  await new Promise(resolve => setTimeout(resolve, randomDelay()));

  /*****
    Use the callback function to output data that can be used in later actions in your workflow.
  *****/
  callback({
    outputFields: {
    }
  });
};

[bryce-corey]

Hey there @spotthehub! Thanks for the reply, I should've updated this with the solution. Support was correct with their suggestion, once I stopped catching the error, and started letting it raise to the top of the stack, the custom code action was able to retry due to the exception. 

[spotthehub]

Thanks for the reply @bryce-corey ... Is this how you implemented the throw?

It's not super clear when the retries are happening, etc. 

try {

// custom code here

} catch (err) {
console.log(err);
throw err;
};