Regarding bulk data import in HubSpot through API

Develop, Build & Integrate APIs & Integrations
, ,
1

We are planning to import 21+ lakhs of data to import through API. We import customers, deals & line items.

Currently, we can import 14,400 records daily. We can successfully import 20 records every 2 minutes through API. We are getting rate limits exceeding if we try to import more records.

We need confirmation on whether HubSpot will support it or not.

i) We will use the AWS lambda function. Several lambda functions will run parallel and process a specific set of data. Do we need different Private App keys for different lambda functions? because we are getting a timeout issue when trying to import more than 20 records every 2 min using one private app key.

ii) As per HS APIs’ use limit, we can make more calls every second. Do we need to make any adjustments on our side so that it will allow frequent calls without the rate limit exceeding?

2

Hey, @SMAITY2 :waving_hand: Thanks for your question.

Considering rate limits is always important.

I have a few questions to help the community better understand your project:

  • Have you put in place some logic to handle the rate limit responses (429 status code) and retries?
  • Are you receiving the 429 status codes?
  • Have you checked the ‘X-HubSpot-RateLimit-Remaining’ header in the response to track how many API calls you have left in the current 10-second window?
  • Can you tell us more about the ‘timeout’ issue you mentioned?
  • Have you tried using any of the bulk endpoints?

Thank you for the additional details. This helps the community give you more specific and targeted suggestions.

Best,

Jaycee