topic Re: Issue with ADF Pipeline Reading Only 100 Contact Records from HubSpot in Sandbox Environment ... in APIs & Integrations

ADF Pipeline with HubSpot Linked Service Only Reading 100 Contact Records from Sandbox (No API Used)

VBhutkar — Fri, 13 Sep 2024 04:45:41 GMT

Hello Everyone,

I have implemented an Azure Data Factory (ADF) pipeline where I’ve set up a linked service connected to HubSpot. All necessary details like client ID, client secret, token, and refresh token were provided, and the connection was successfully established. I am not using any direct API calls in the pipeline, just the built-in linked service functionality.

When I select the "HubSpot.Contacts" table from the tables dropdown, the pipeline only retrieves 100 records, even though there are more than 100,000 contact records in the sandbox environment.

What’s confusing is that when I use the same pipeline to connect to another client’s HubSpot environment (not sandbox), it successfully reads 7,000+ contact records in one go without any issues.

Initially, the pipeline in the sandbox environment returned 14,000+ records, but now it consistently returns only 100 records. I’m unsure what might have changed, and I haven’t altered the pipeline setup.

Could there be any limitations or differences in configuration between the environments that might explain this behavior? Has anyone faced a similar issue?

Any help or insights would be greatly appreciated. Thank you!

Re: Issue with ADF Pipeline Reading Only 100 Contact Records from HubSpot in Sandbox Environment ...

Jaycee_Lewis — Fri, 13 Sep 2024 15:21:58 GMT

Hey, @VBhutkar 👋 Thanks for your post. I've noticed ADF questions can be tough to troubleshoot via the community (you definitely didn't do anything wrong by posting here) Hey @johnelmer has a well-regarded app in the marketplace and might have a tip or suggestion for you.

Thanks for taking a look! — Jaycee

Re: Issue with ADF Pipeline Reading Only 100 Contact Records from HubSpot in Sandbox Environment ...

VBhutkar — Sun, 15 Sep 2024 13:34:19 GMT

Thank you, @Jaycee_Lewis for your response.

However, I would specifically like to ask HubSpot Support to confirm if there are any limitations in the sandbox environment related to the number of records returned when using Azure Data Factory (ADF) linked services.

As I mentioned in my initial post, I’ve encountered a situation where the same ADF pipeline retrieves more than 7,000 records from another production HubSpot environment, but in the sandbox, it consistently retrieves only 100 records.

Could you confirm if there’s any limit on the number of records retrieved in the sandbox, or if there’s any configuration difference that might explain this behavior?

Any help or insights would be greatly appreciated. Thank you!