414 URI too long with Azure Hubspot Connector

Developer Support APIs & Integrations
1

I am in need of help with a 414 error while pulling data from Hubspot using the Azure Data Factory - Hubspot Connector.

What am I doing

I am trying to build a pipeline in Azure Data Factory that loads data from Hubspot into an Azure SQL Database. I am using the out-of-the-box connector to connect to Hubspot that ADF provides and I used this Microsoft guide. For development I’ve set up an Hubspot Developers Account and use a Hubspot Test Account with fake data. I’ve set up a dynamic pipeline and all worked fine while testing it on the Test Account. But now that I’ve connected the pipeline to our Hubspot production environment, it fails for some Hubspot tables.

The problem

The specific error I get is a “414 Request-URI Too Large” error. I get this when pulling data from the table “Hubspot”.”Contacts_Properties_Values” through the Hubspot connector. I almost directly understand why this error is happing. Probably because in our Hubspot production environment we have 100+ custom properties and every property is automatically being put in the request uri by the connector. Since my Hubspot Developer Account only has 6 custom properties, this probably isn’t to much to handle.

Solution ideas

When looking online I find many people with the same issue whey using API’s, but no information of other users that use the Hubspot connector. I did find this community post which offers a possible solution I hope Microsoft is willing to add to their Hubspot connector.

2

Hi @NDam,

Thank you for reaching out to the Community and describing the steps you’re taking.

It would be great if you’d reach out to Microsoft as well since the Data Factory connector is built/configured by them and we don’t have insight into how it is built.

I still want to provide some helpful information – I had a look in our documentation and found the following recommendations on how to resolve this error:

Since the issue here is that the request URL is too large, adjusting the API request will likely solve the issue. Instead of including every single property in the request URL, the integration provider could limit the request to current properties only (PropertiesFetchMode=latest_version). This only works for the v1 legacy APIs.

We do also have the V3 CRM search API which allows you to list all of the properties in the POST body rather than submitting them as query parameters.

To be transparent, I’m not a developer, so I won’t be able to provide more detailed information or insight on the above, but I hope this will be helpful for the Microsoft team when you partner with them to resolve the error.

Cheers
Mia, Community Team