⚙ Operations Hub

ChrisoKlepke
Key Advisor | Elite Partner
Key Advisor | Elite Partner

429 API Rate Limit Error

SOLVE

Hey RevOps-Gang, 

 

hope you're all doing well. I encountered a problem in a Custom Code Action recently which I thought HubSpot took care of, on its own, but I might be wrong. I have a Python-based action that does multiple requests against the HubSpot API. Some requests cannot be done with the api-library (e.g. HubDB stuff with filtering and sorting) so I use the requests library for that. 

 

As mentioned in the documentation here

If the call fails due to a rate limiting error, or a 429 or 5XX error from axios or @hubspot/api-client, HubSpot will reattempt to execute your action for up to three days, starting one minute after failure. Subsequent failures will be retried at increasing intervals, with a maximum gap of eight hours between tries

 

I'm not entirely sure, but this should also mean that calls with the requests library are also retried if it encounters a rate limit error, correct?

 

Most of these calls follow this pattern with try and except statements:

 

try:
getPlantContent = requests.get(
"https://api.hubapi.com/cms/v3/hubdb/tables/plant_content/rows?" + plantContentProperties + plantContentFilters + "&hapikey=" + os.getenv(
"HAPIKEY")).json()["results"][0]["values"]

except ApiException as e:
print("Exception when calling rows_api->get_table_rows: %s\n" % e)

 

Now, I've noticed that this is missing the raise keyword in the except block. Is this of importance? If so, what does it do? The only info I found is that it re-raises the exception if there is no other argument associated with raise. 

 

Thank you all for your help! Please be easy on me since I'm not a trained dev.

 

Cheers, 

Chriso

0 Upvotes
1 Accepted solution
tfoston01
Solution
Participant | Elite Partner
Participant | Elite Partner

429 API Rate Limit Error

SOLVE

The documentation is wrong.

 

From experience, the SDK will retry a failed API call 6 times max. After that, it stops retrying and you're on your own to essentially restart things yourself.

 

At A8Labs, we actually don't use the SDK and instead make the API calls ourselves using 

the axios library. We evaluate the response headers and look for the rate-limiting headers specifically and take pauses/sleep if we can anticipate a 429 coming. 

 

For job queueing and decoupling, please check out the article below

https://towardsaws.com/aws-sqs-the-ultimate-decoupling-solution-de8a8c8862e7

 

View solution in original post

6 Replies 6
tfoston01
Participant | Elite Partner
Participant | Elite Partner

429 API Rate Limit Error

SOLVE

Looking at this phrase....

 

If the call fails due to a rate limiting error, or a 429 or 5XX error from axios or @hubspot/api-client, HubSpot will reattempt to execute your action for up to three days, starting one minute after failure. Subsequent failures will be retried at increasing intervals, with a maximum gap of eight hours between tries

 

Unless your script is also going to run long to take full advantage of the long time span of re-trying you shouldn't count on this. When you're running into persistent API rate limiting issues, you'll want to consider some level of decoupling to your flow. I would suggest looking into making a job queue to break up your processes up so that you're better able to regulate the API calls

ChrisoKlepke
Key Advisor | Elite Partner
Key Advisor | Elite Partner

429 API Rate Limit Error

SOLVE

Hey @tfoston01 , 

 

Thank you very much for getting in on this. I was aware of this sentence in the documentation. What I'm confused by is the wording of that sentence. It seems to me the code action should retry itself if one of the 3 scenarios applies. 

  • call fails due to a rate limiting error,
  • or a 429 or 5XX error from axios 
  • or @hubspot/api-client,

If I use the requests library and run into a 429 I think the first bullet applies, but in no instance got the action retried. That's why I suspected something wrong with my code, or more specifically the exception, since it also says in the documentation:

If you're using Python and encounter a rate limiting error but you want HubSpot to retry your call, you'll need to raise the error in the except block of your custom code action.

 

So, either I'm doing something wrong, or the documentation is wrong in that, that the retries are not supported with the requests library. What are your thoughts on this?

 

In any case, you mentioned a job queue to get around all of that. How would someone create something like that in the custom code action? Is there something you can point me to?

 

Cheers, 

Chriso

tfoston01
Solution
Participant | Elite Partner
Participant | Elite Partner

429 API Rate Limit Error

SOLVE

The documentation is wrong.

 

From experience, the SDK will retry a failed API call 6 times max. After that, it stops retrying and you're on your own to essentially restart things yourself.

 

At A8Labs, we actually don't use the SDK and instead make the API calls ourselves using 

the axios library. We evaluate the response headers and look for the rate-limiting headers specifically and take pauses/sleep if we can anticipate a 429 coming. 

 

For job queueing and decoupling, please check out the article below

https://towardsaws.com/aws-sqs-the-ultimate-decoupling-solution-de8a8c8862e7

 

ChrisoKlepke
Key Advisor | Elite Partner
Key Advisor | Elite Partner

429 API Rate Limit Error

SOLVE

Thank you for the insight @tfoston01 . I'm not that familiar with JS and/or axios but if that's the limitation for Python and the requests library right now I will give this issue to more qualified people than me. 

 

@MiaSrebrnjak, it might be a good idea to check the documentation with the responsible team again and change it if it proves to be confusing.

 

Maybe @jackcoldrick or @jbogaert are the right people for this?

MiaSrebrnjak
Community Manager
Community Manager

429 API Rate Limit Error

SOLVE

Hi @ChrisoKlepke, thank you for flagging this! I'll send it to our technical writers for a review!

 

Cheers
Mia, Community Team 


Wusstest du, dass es auch eine DACH-Community gibt?
Nimm an regionalen Unterhaltungen teil, indem du deine Spracheinstellungen änderst


Did you know that the Community is available in other languages?
Join regional conversations by
changing your language settings

0 Upvotes
MiaSrebrnjak
Community Manager
Community Manager

429 API Rate Limit Error

SOLVE

Hi @ChrisoKlepke,

 

Thank you for reaching out to the Community!

 

I want to tag in some subject matter experts to see if they have any suggestions.

Hi @JBeatty@taran42@A_Wessolly, do you have any thoughts on this? Thank you!! 

 

 

Cheers
Mia, Community Team


Wusstest du, dass es auch eine DACH-Community gibt?
Nimm an regionalen Unterhaltungen teil, indem du deine Spracheinstellungen änderst


Did you know that the Community is available in other languages?
Join regional conversations by
changing your language settings