topic Re: Help - API throttling? in APIs & Integrations

Help - API throttling?

netlify-data — Mon, 15 Mar 2021 11:26:33 GMT

Hi,

We’ve got Fivetran set up to pull HubSpot data into our data warehouse. It’s been having problems for 1 week where syncs take hours and hours. I feel we’re being throttled on our API usage, even though I know we’re sending few API requests. We get these errors in Fivetran’s logs:

"Some Error occurred, Retrying after 60 seconds. Error : HTTP 500 Internal Server Error"
"Retrying after 60 seconds, Reason : Bad Gateway"

Where can we get some insight into this issue and whether our API usage is being throttled or blocked?

Re: Help - API throttling?

dennisedson — Mon, 15 Mar 2021 19:24:34 GMT

Hi @netlify-data

We should add @Fivetran to the conversation 😀

Re: Help - API throttling?

netlify-data — Tue, 16 Mar 2021 10:21:18 GMT

Sure, we're also talking with @fivetran via their support system - if it helps for @fivetran this is the link to the conversation over there. It seems the @fivetran engineers still investigating what's happening either so I'm trying to rule out any API throttling from HubSpot's end and identify whether the origin of this issue is with HubSpot or Fivetran.

@dennisedson are you able to give us any insight into whether we are being rate-limited?

Re: Help - API throttling?

dennisedson — Tue, 16 Mar 2021 15:27:54 GMT

@netlify-data ,

Just did a quick check on your usage and you seem to be within the bounds. (current usage 6156, remaining: 493844) and this is a daily limit that resets at midnight your timezone. Keep us in the loop on this.

@lynton , any insights that you can draw here? (If this is not a fivetran issue, I feel like this one might only be solvable by HubSpot, but thought it wouldn't hurt to have you eyes on it)

Re: Help - API throttling?

lynton — Wed, 17 Mar 2021 00:19:28 GMT

A Bad Gateway error does happen occasionally during heavy api load. Fivetran should be able to recover gracefully and continue retrying when encountering these errors. Keep pushing their support team for a resolution as they should be able to work around the network errors.

Re: Help - API throttling?

netlify-data — Wed, 17 Mar 2021 09:51:38 GMT

Thanks everyone, I'll do that! 👍

Re: Help - API throttling?

diegofischer — Sat, 14 Aug 2021 10:02:38 GMT

I have the same issue. What was the solution?
(I'm using Fivetran to sync data from Hubspot)

{
"event":"warning",
"data":{
"type":"retry_api_call",
"message":"Retrying after 60 seconds, Reason : Too many requests"
},
"created":"2021-08-14T09:51:47.871Z",
"connector_type":"hubspot",
"connector_id":"hubspot",
"connector_name":"hubspot"
}

My daily and secondly limits are within boundaries though as you can see:

X-HubSpot-RateLimit-Daily: 1000000

X-HubSpot-RateLimit-Daily-Remaining: 837628

X-HubSpot-RateLimit-Max: 200

X-HubSpot-RateLimit-Remaining: 130

X-HubSpot-RateLimit-Secondly: 20

X-HubSpot-RateLimit-Secondly-Remaining: 13

Thank you