get all contacts

[PowerMyAnalytic]

Hi HubSpot,

I have a problem with fetching all contacts.

My client have 208,000 contacts. 

Fetch it will take 30 minutes with pagination (with count limit of 100) - but I work with google functions so I get timeout after 10 minuets.

I think of 2 possible solutions - 

1. enlarge count to 1000.

2. maybe you can provide the option to fetch contacts by date range,

so I can split to tasks from my side.

Thanks

[Derek_Gervais]

Hey @PowerMyAnalytic ,

Can you give me more details on your process? At the theoretical maximum rate, pulling 208,000 contacts should only take ~3 minutes. Is there some processing you're doing that's causing it to take longer?

Also, any context around what exactly you're doing with the contacts would be helpful too. How frequently are you pulling all of the contacts in your database? Is it possible for you to use the 'Get recently modified' or 'Get recently created' endpoints instead? (see below)

https://developers.hubspot.com/docs/methods/contacts/get_recently_updated_contacts

https://developers.hubspot.com/docs/methods/contacts/get_recently_created_contacts

[PowerMyAnalytic]

Hey Derek_Gervais,

Thanks for fast answer 🙂

Just regular pagination...

count = 100

if 'hes-more' === true

             next = request: vidOffset = 'vid-offset'

It takes 21 minutes...

I will use it only in the first time and then use 'Get recently modified'.

But, still, I have 10 minutes timeout.

[Derek_Gervais]

Hey @PowerMyAnalytic ,

So here's where my figure came from:

208,000 contacts at 100 contacts per request means you need to make 2080 requests.

2080 requests at 10 requests per second will take 208 seconds, which is ~3.5 minutes. 

Does your app/script do some sort of processing on the request body before making the next request? It sounds like you might not be making requests as fast as you could be, and my gut tells me that might be the reason. Is it possible for you to pull the contacts & queue them up for processing in a separate process? This would free you up to pull all 208,000 contacts within the 10 minute timeframe. From there, it'd be a matter of optimizing your processing to complete within the remining ~7 minutes.