APIs & Integrations

nmfoss
Participant

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi again,

 

I am currently creating a API connector to extract data from HubSpot to a datawarehouse solution.

 

We are using the GET /crm/v3/objects/<object> endpoints to do most of the extraction. Here we also ask for all associations for each object to any object. These associations also come with separate pagination links. We handle this.

 

My question is: we see that when running a full load using these endpoints we end up missing a bunch of associations. We are certain we are picking up all associations on "first page" (together with the object properties) and all subsequent pagination pages for each type of association.

 

When we have identified some missing associations we can make a call to the associations endpoint, eg:

GET api.hubapi.com/crm/v4/objects/companies/xxxxx/associations/emails

and here we can find the missing associations. So there is no scoping or user settings restricting the associations.

 

Is there somekind of undocumented features of how the associations works in the objects endpoints that we are using? For instance there is no documentation on the pagination for these associations, so is there something else we should be aware of that might explain the missing associations?

 

Sincerely,

NM Foss

2 Accepted solutions
PCronin9
Solution
Contributor

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi @nmfoss, I agree with you, this is a great question, and I have it too! .If you have an object with more than 100 associations, there _used_ to be an endpoint in the v3 Associations API that would facilitate you paginating the remainder of the associations, but it appears that endpoint has been removed from the documentation and crippled (but not removed!) in functionality (see details in my linked post).

 

For our organization, we don't have an object with over 100 associations yet, so I haven't had to cross this bridge yet. But, if I had to solve it today, I would use the V4 Associations API (either the Batch - Read or Basic - List endpoints) to get all the associations I need. Do note however that the pagination information from the V3 CRM API is not compatible with the V4 Associations API, so if you do this, I'd throw out the ones you got on the "first page" of results from the V3 CRM API (or simply not request them at all), and use the V4 Associations API to get them all.

 

For what it's worth, I find this behavior of the HubSpot APIs quite alarming, and I find the lack of participation, comment, and interest in bug reports from anyone employed by HubSpot to be simiarly alarming. In my experience working with the HubSpot APIs that I do, the developer experience has been quite weak, and I have not found anyone at HubSpot yet who seems to share my concerns with what appear to be genuine bugs in their APIs.

View solution in original post

0 Upvotes
BérangèreL
Solution
Community Manager
Community Manager

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi @PCronin9 and @nmfoss,

Thanks for your valuable feedback. This means a lot to us.
I have shared it internally for more visibility.

I'm sorry to hear about your experience regarding this.

I would like to apologize for the frustration, data issues, and business impact this has caused.

Thank you @PCronin9 also for sharing this workaround that can help in the meanwhile.

I have checked internally for you and I could see that HubSpot will soon be releasing a V4 of the CRM Object API that will help for this.
I don't have an exact timeline to share for this at the moment but the Team is actively working on it as we speak.

Thank you for your patience.

Have a great day!

Best,
Bérangère


Saviez-vous que la Communauté est disponible en Français ?
Rejoignez les discussions francophones en changeant votre langue dans les paramètres !

Did you know that the Community is available in other languages?
Join regional conversations by changing your language settings!


View solution in original post

4 Replies 4
GKurtulmus
Member

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

I use the following format to be able to retrieve all the data for a given object:

 

function.basic_api.get_page(
limit=100,
properties=properties,
associations = associations,
archived=False,
after=after)
 
For this purpose I use pagination with the value at ''response.paging.next.after'' and add it to the 'after'.

But if there are associations that specifically require pagination, there are new keys for each of the association type. Named as
'associations.contacts.paging.next.after' or

'associations.companies.paging.next.after'.

How should I query them to get all the associations for the given record?



0 Upvotes
PCronin9
Solution
Contributor

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi @nmfoss, I agree with you, this is a great question, and I have it too! .If you have an object with more than 100 associations, there _used_ to be an endpoint in the v3 Associations API that would facilitate you paginating the remainder of the associations, but it appears that endpoint has been removed from the documentation and crippled (but not removed!) in functionality (see details in my linked post).

 

For our organization, we don't have an object with over 100 associations yet, so I haven't had to cross this bridge yet. But, if I had to solve it today, I would use the V4 Associations API (either the Batch - Read or Basic - List endpoints) to get all the associations I need. Do note however that the pagination information from the V3 CRM API is not compatible with the V4 Associations API, so if you do this, I'd throw out the ones you got on the "first page" of results from the V3 CRM API (or simply not request them at all), and use the V4 Associations API to get them all.

 

For what it's worth, I find this behavior of the HubSpot APIs quite alarming, and I find the lack of participation, comment, and interest in bug reports from anyone employed by HubSpot to be simiarly alarming. In my experience working with the HubSpot APIs that I do, the developer experience has been quite weak, and I have not found anyone at HubSpot yet who seems to share my concerns with what appear to be genuine bugs in their APIs.

0 Upvotes
BérangèreL
Solution
Community Manager
Community Manager

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi @PCronin9 and @nmfoss,

Thanks for your valuable feedback. This means a lot to us.
I have shared it internally for more visibility.

I'm sorry to hear about your experience regarding this.

I would like to apologize for the frustration, data issues, and business impact this has caused.

Thank you @PCronin9 also for sharing this workaround that can help in the meanwhile.

I have checked internally for you and I could see that HubSpot will soon be releasing a V4 of the CRM Object API that will help for this.
I don't have an exact timeline to share for this at the moment but the Team is actively working on it as we speak.

Thank you for your patience.

Have a great day!

Best,
Bérangère


Saviez-vous que la Communauté est disponible en Français ?
Rejoignez les discussions francophones en changeant votre langue dans les paramètres !

Did you know that the Community is available in other languages?
Join regional conversations by changing your language settings!


BérangèreL
Community Manager
Community Manager

Using GET /crm/v3/objects/<object> endpoints for extracting associations

SOLVE

Hi @nmfoss,

Thank you for asking the Community!

I believe there may be a limit in the number of rows you can get. Which API documentation did you use, please? Is there any limit mentioned on it?

Have you tried breaking up your call in several requests to get all associations, please?

I also wanted to invite a couple of subject matter experts and some Community Members to this conversation: Hi @alyssamwilie, @jonchim, @PCronin9, @HBrandao and @stefen do you have suggestions to help @nmfoss, please?

If anybody else has anything to add and/or share, please feel free to join in the conversation 🙂

Thanks a lot and have a wonderful day!

Best,
Bérangère


Saviez-vous que la Communauté est disponible en Français ?
Rejoignez les discussions francophones en changeant votre langue dans les paramètres !

Did you know that the Community is available in other languages?
Join regional conversations by changing your language settings!


0 Upvotes