Having trouble getting all contacts 1 at a time

[mgrotton]

I'm writing some code that processes Contacts one at a time.  My initial call looks like this:

'https://api.hubapi.com/contacts/v1/lists/all/contacts/recent?hapikey=' . $hap_key . '&count=1&formSubmissionMode=all';

if has-more is set, I use the vid-offset returned like so:

'https://api.hubapi.com/contacts/v1/lists/all/contacts/recent?hapikey=' . $hap_key . '&count=1&formSubmissionMode=all&vidOffset=' . $vid_offset;

It is looping through users just fine, the problem is I have around 17,000 and by the time has-more becomes 0, only around 2,200 have been processed.  

Any ideas?

[cbarley]

Hi @mgrotton , it looks like you're using the get recently updated endpoint rather than the get all contacts endpoint. The get recently updated endpoint only scrolls back in time 30 days. If you want to get all contacts, you should use the second endpoint I linked. I would also recommend getting contacts 100 at a time (the max) so that you use less API calls.

[mgrotton]

Hi, the endpoint I posted about IS the get all endpoint.  We're using the other in our script for 'catch up' runs, but those are meant to look at contacts less than 30 days old.

Also, since writing this question  I've run the same process twice with no changes and the second time it actually did complete the scan.

Can't go with 100, because we are using data from the contacts to get information from another API and the combined processing time is long enough that processing more than a couple at a time would cause our PHP end of the process to time out.

[cbarley]

Hi @mgrotton , apologies, I linked the wrong endpoint in the last message. However, the endpoint you posted https://api.hubapi.com/contacts/v1/lists/all/contacts/recent is to get recently created contacts: see the docs here. The docs state that if you're using this endpoint you need to use the vid-offset in conjunction with the time-offset parameters on each subsequent call. 

I'd opt to use the get all contacts endpoint, which is a different URL than the one you posted. That endpoint looks like this: https://api.hubapi.com/contacts/v1/lists/all/contacts/all

I just ran my own portal through a tool I created and got all 1000 contacts back going 1 at a time: https://github.com/cbarley10/hubspot-pagination so it looks like that endpoint should be functioning correctly!

We did have an issue recently that affected the APIs, but should now be resolved: https://status.hubspot.com/incidents/f8zzfyc5cs1c

Edit: I see why you're using the recent endpoint if you only want to go back 30 days, but make sure you're also using the time-offset property as well as described by the docs on that endpoint