Deals API pagination

[Adaptive]

Hi all,

I have created some integration already using the Contacts and Contacts List API.

Now, I have a requirement to work with the Deals API as well. Follow the problem I am facing -

Hubspot API does not allow filters in it's requests. Which means I would need to retrieve ALL contacts in Hubspot (every time I want to update the information on my side) and it's relevant properties, then I can separate information on my side.

For Contacts we can create contacts lists, though. This allows us filtering them using some properties and then fetch the data within this contact list using the contact list API: 

• https://developers.hubspot.com/docs/methods/lists/get_list_contacts 

Using dates and properties filters to setup a bunch of lists, I don't need to go through all contacts everytime I want to update my side. This worked fantasticly for my last automation.

Now, more information (Deals) are also required and I can't find any workaround for this.
I was hoping have the same ability with the deals, creating pipelines instead of lists, for example, but the API doesn't have the option to "return deals inside a pipeline" endpoint.
Follow what we have -

Fetch deals:
https://developers.hubspot.com/docs/methods/deals/get-all-deals  

Fetch pipelines:

https://developers.hubspot.com/docs/methods/pipelines/pipelines_overview  

It doesn't seem to have anything like this:

https://developers.hubspot.com/docs/methods/lists/get_list_contacts 

This is a problem because:

I need to update an excel (weekly internal report) with, for example, the information:

Date                   Origin           Country        Leads    Sales    Revenue

06/06/2019      Website        Ireland          10          6           60

Instead of fecthing all contacts and then separating them every week, I found easier to create contact lists where I can set:

Date is after: 01/06/2019

Origin is : Website

Country is: Ireland

And then fetch the contacts in this contacts list, retrieving the property "country". So I can separate it.

When it comes to deals, it doesn't seem to have the same option. The only option I have is to use this:

https://developers.hubspot.com/docs/methods/deals/get-all-deals

and paginate it, every week, until it reachs the end. Then do the same again next week. It tends to be undoable overtime.

I was thinking about saving the last pagination and every week the code only runs after the last pagination but:

• I am not sure if my last deals will be always at the end of the entire list (after my last pagination from last week)
• I cannot find the last pagination id before going through this entire list at least once.

Any ideas on how to work around this without need to fetch every week the whole amount of deals within Hubspot?

The limitation of 250 entities for every requisition might become a problem soon if I need to fecth everything every week.

Thank you,

[cbarley · Accepted solution]

Hi @Adaptive , we definitely feel your pain here. Over the coming months we'll be looking ot add filtering to all of our APIs, but the issue is that some of our older APIs, like deals, just can't support that without a major overhaul. I think you're doing everything correctly here, and apologies if there's no really great solution. The only thing I can think of to help you is either to use the recently created deals endpoint here: https://developers.hubspot.com/docs/methods/deals/get_deals_created, or listen to deal creations in your account by using the Webhooks API in conjunction with your already-existing processes. This way, you can listen for deal creations, and not have to page through all deals to get what you need.

[Adaptive]

Thank you @cbarley!

I will be waiting for the new version of the Deals API with filters then and use the current work around for now.

Thank you.