⚙ Operations Hub

ChrisoKlepke
Conselheiro(a) de destaque | Parceiro Elite
Conselheiro(a) de destaque | Parceiro Elite

429 API Rate Limit Error

resolver

Hey RevOps-Gang, 

 

hope you're all doing well. I encountered a problem in a Custom Code Action recently which I thought HubSpot took care of, on its own, but I might be wrong. I have a Python-based action that does multiple requests against the HubSpot API. Some requests cannot be done with the api-library (e.g. HubDB stuff with filtering and sorting) so I use the requests library for that. 

 

As mentioned in the documentation here

If the call fails due to a rate limiting error, or a 429 or 5XX error from axios or @hubspot/api-client, HubSpot will reattempt to execute your action for up to three days, starting one minute after failure. Subsequent failures will be retried at increasing intervals, with a maximum gap of eight hours between tries

 

I'm not entirely sure, but this should also mean that calls with the requests library are also retried if it encounters a rate limit error, correct?

 

Most of these calls follow this pattern with try and except statements:

 

try:
getPlantContent = requests.get(
"https://api.hubapi.com/cms/v3/hubdb/tables/plant_content/rows?" + plantContentProperties + plantContentFilters + "&hapikey=" + os.getenv(
"HAPIKEY")).json()["results"][0]["values"]

except ApiException as e:
print("Exception when calling rows_api->get_table_rows: %s\n" % e)

 

Now, I've noticed that this is missing the raise keyword in the except block. Is this of importance? If so, what does it do? The only info I found is that it re-raises the exception if there is no other argument associated with raise. 

 

Thank you all for your help! Please be easy on me since I'm not a trained dev.

 

Cheers, 

Chriso

0 Avaliação positiva
1 Solução aceita
tfoston01
Solução
Participante | Parceiro Elite
Participante | Parceiro Elite

429 API Rate Limit Error

resolver

The documentation is wrong.

 

From experience, the SDK will retry a failed API call 6 times max. After that, it stops retrying and you're on your own to essentially restart things yourself.

 

At A8Labs, we actually don't use the SDK and instead make the API calls ourselves using 

the axios library. We evaluate the response headers and look for the rate-limiting headers specifically and take pauses/sleep if we can anticipate a 429 coming. 

 

For job queueing and decoupling, please check out the article below

https://towardsaws.com/aws-sqs-the-ultimate-decoupling-solution-de8a8c8862e7

 

Exibir solução no post original

8 Respostas 8
Unmatched
Participante

429 API Rate Limit Error

resolver

Hey! We've built an app that will be able to help you with this. It's called Time Turner, and it allows you to throttle your HubSpot workflows and drip-feed your emails out over time. 

 

Let me know what you think? 

 

Here's the install link for Time Turner!

0 Avaliação positiva
tfoston01
Participante | Parceiro Elite
Participante | Parceiro Elite

429 API Rate Limit Error

resolver

Looking at this phrase....

 

If the call fails due to a rate limiting error, or a 429 or 5XX error from axios or @hubspot/api-client, HubSpot will reattempt to execute your action for up to three days, starting one minute after failure. Subsequent failures will be retried at increasing intervals, with a maximum gap of eight hours between tries

 

Unless your script is also going to run long to take full advantage of the long time span of re-trying you shouldn't count on this. When you're running into persistent API rate limiting issues, you'll want to consider some level of decoupling to your flow. I would suggest looking into making a job queue to break up your processes up so that you're better able to regulate the API calls

ChrisoKlepke
Conselheiro(a) de destaque | Parceiro Elite
Conselheiro(a) de destaque | Parceiro Elite

429 API Rate Limit Error

resolver

Hey @tfoston01 , 

 

Thank you very much for getting in on this. I was aware of this sentence in the documentation. What I'm confused by is the wording of that sentence. It seems to me the code action should retry itself if one of the 3 scenarios applies. 

  • call fails due to a rate limiting error,
  • or a 429 or 5XX error from axios 
  • or @hubspot/api-client,

If I use the requests library and run into a 429 I think the first bullet applies, but in no instance got the action retried. That's why I suspected something wrong with my code, or more specifically the exception, since it also says in the documentation:

If you're using Python and encounter a rate limiting error but you want HubSpot to retry your call, you'll need to raise the error in the except block of your custom code action.

 

So, either I'm doing something wrong, or the documentation is wrong in that, that the retries are not supported with the requests library. What are your thoughts on this?

 

In any case, you mentioned a job queue to get around all of that. How would someone create something like that in the custom code action? Is there something you can point me to?

 

Cheers, 

Chriso

tfoston01
Solução
Participante | Parceiro Elite
Participante | Parceiro Elite

429 API Rate Limit Error

resolver

The documentation is wrong.

 

From experience, the SDK will retry a failed API call 6 times max. After that, it stops retrying and you're on your own to essentially restart things yourself.

 

At A8Labs, we actually don't use the SDK and instead make the API calls ourselves using 

the axios library. We evaluate the response headers and look for the rate-limiting headers specifically and take pauses/sleep if we can anticipate a 429 coming. 

 

For job queueing and decoupling, please check out the article below

https://towardsaws.com/aws-sqs-the-ultimate-decoupling-solution-de8a8c8862e7

 

dthreatt
Membro

429 API Rate Limit Error

resolver

It just doesnt work.  I get rate limit errors on workflows that have no APIs or custom codes.  Its ridiculous.

0 Avaliação positiva
ChrisoKlepke
Conselheiro(a) de destaque | Parceiro Elite
Conselheiro(a) de destaque | Parceiro Elite

429 API Rate Limit Error

resolver

Thank you for the insight @tfoston01 . I'm not that familiar with JS and/or axios but if that's the limitation for Python and the requests library right now I will give this issue to more qualified people than me. 

 

@MiaSrebrnjak, it might be a good idea to check the documentation with the responsible team again and change it if it proves to be confusing.

 

Maybe @jackcoldrick or @jbogaert are the right people for this?

MiaSrebrnjak
Gerente da Comunidade
Gerente da Comunidade

429 API Rate Limit Error

resolver

Hi @ChrisoKlepke, thank you for flagging this! I'll send it to our technical writers for a review!

 

Cheers
Mia, Community Team 




Wusstest du, dass es auch eine DACH-Community gibt?
Nimm an regionalen Unterhaltungen teil, indem du deine Spracheinstellungen änderst


Did you know that the Community is available in other languages?
Join regional conversations by
changing your language settings


0 Avaliação positiva
MiaSrebrnjak
Gerente da Comunidade
Gerente da Comunidade

429 API Rate Limit Error

resolver

Hi @ChrisoKlepke,

 

Thank you for reaching out to the Community!

 

I want to tag in some subject matter experts to see if they have any suggestions.

Hi @JBeatty@taran42@A_Wessolly, do you have any thoughts on this? Thank you!! 

 

 

Cheers
Mia, Community Team




Wusstest du, dass es auch eine DACH-Community gibt?
Nimm an regionalen Unterhaltungen teil, indem du deine Spracheinstellungen änderst


Did you know that the Community is available in other languages?
Join regional conversations by
changing your language settings