Get all 1M+ contacts via API

[adley_amzovski_]

Our HubSpot account has over 1million+ contacts & we want to retrieve all over the API v3 - which has a limit of 100 per call. Meaning, I would need to run the loop for 10K times & calculating that accounts to ~8days for retrival.

I would have parallelized the calls but then the next api call is dependent on the pagination.next.after value from the previous api call - so that's not an option (unless there's a work around / calculation logic)

Moreover, I could have run that for the initial load but then I couldn't find an API mechnaism that would return the existing contacts that were updated recently. There's an API to pull the recent contacts but that would pull the newly added contacts only. I'm looking for a method to perform CDC post initial extract.

An example to achieve that will be the ability to pass time as a filter in the API call.

Any insights are appreciated!

[TAfolabi4]

Good Afternoon, 

I am new to hubspot and api calls. I used the get all contacts ('https://api.hubapi.com/contacts/v1/lists/all/contacts/all?hapikey =x&count =40000) . I only received 500 contacts, but I know I have over 30k contacts in hubspot. I see vid offset in the contacts api documentation, but not sure how to use it. Do I need to do pagination or vid to get the remaining contacts? Please can you show or refer me to a video example.

Regards,

Tola

[WendyGoh · Accepted solution]

Hey @adley_amzovski_ 

When looking to pass time i.e. created date or last modified date, you can use the CRM API | Search. 

An example:

POST crm/v3/objects/contacts/search

{
    "filterGroups":[
      {
        "filters":[
          {
            "propertyName": "lastmodifieddate",
            "operator": "GT",
            "value": "1583769599999"
          }
        ]
      }
    ]
  }

[adley_amzovski_]

Hi @WendyGoh

Thanks for the response. This would work fine for getting the records filtered by "lastmodifieddate" property.

Is there a better approach as such for the intial download of all the contacts? There is a bulk-import API for CRM data and looking for something similar to do like a bulk-export?

Thank you in advance!

[WendyGoh]

Hey @adley_amzovski_,

For now, there isn't a way to bulk export all contacts.

The get all contacts endpoint to retrieve 100 at a time is probably the best way around. 

If you'd like to bulk export, one suggestion that I have is for your team to do a one time export from the UI - Export contacts, companies, deals or tickets > save it on your external database > for the subsequent new contacts you can use the search v3 endpoint to retrieve new incoming contacts. 

[adley_amzovski_]

Hi @WendyGoh 

Thanks for sharing that. I had read this blog recently & had been meaning to test that but just wanted to check if there's a better programmatic approach made available. But I guess, we should be fine with one of the either available options for now.