topic Re: API Contacts: Remote end closed connection without response in APIs & Integrations

API Contacts: Remote end closed connection without response

camera — Sun, 16 Jan 2022 02:49:15 GMT

Hello people!

I'm doing some requests to the Contacts endpoint to get all of my contacts. But, sometimes I get an error that I can't handle. I think that the server isn't responding to my request.

There are a lot of contacts (700k) and I'm running the code below in a for iteration.

I know about the rate limits, and I'm sure I'm not passing thru.

url = 'https://api.hubapi.com/crm/v3/objects/contacts' headers = { 'Content-Type': 'application/json', 'Authorization': 'Bearer ' + <token> } params = { 'limit': 100, 'properties': <list of properties>, 'after': <next_paging number> } response = requests.get(url=url, headers=headers, params=params)

Sometimes I get the error below, sometimes I don't.

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) http.client.RemoteDisconnected: Remote end closed connection without response urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

I've summarized the error so it doesn't get too big.

Can anyone help me, please?

Re: API Contacts: Remote end closed connection without response

dennisedson — Tue, 18 Jan 2022 16:17:33 GMT

@camera

From my searches, this appears to be a python bug perhaps 🤔

@JBeatty , @BJacobson either of you run into this before?

Re: API Contacts: Remote end closed connection without response

camera — Thu, 20 Jan 2022 17:59:27 GMT

Hi @dennisedson !

Thanks for reply! It's not a python bug. It's about the non-response of some messages by Hubspot's API server.

I made some improvements in the API request in the python script. I configured the retry factor from the urllib3 library (https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#module-urllib3.util.retry) and now my pipeline it's not crashing anymore.

But, even so, I would like to know why the Hubspot API server is not answer some of my requests...

Thanks in advance.

Re: API Contacts: Remote end closed connection without response

SBurwash — Wed, 24 Aug 2022 19:15:58 GMT

I came accross this issue also; would there be a way to solve it? Our syncs take a long time since we need to iterate through all of our associations, and this is making our ETL tool fail

Re: API Contacts: Remote end closed connection without response

GAlinio — Mon, 14 Oct 2024 18:14:23 GMT

I'm encountering the same issue today:

ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Even after using the provided sample from here: https://github.com/HubSpot/sample-apps-retry-middleware/blob/main/python/cli.py, the error persists, particularly with status 429 or "RemoteDisconnected."

Here’s my code:

from urllib3.util import Retry from requests.adapters import HTTPAdapter import requests from hubspot import HubSpot # Create a HubSpot client with retry logic def create_client(api_key): # Set interval based on environment if environment == "sb": # Sandbox interval = 6 # 6 seconds for free plan else: # Production interval = 1 # 1 second for pro plan # Define the retry strategy retry_strategy = Retry( total=5, # Total number of retries backoff_factor=interval, # Exponential backoff status_forcelist=[429, 500, 502, 504], # Status codes to retry allowed_methods=["HEAD", "GET", "OPTIONS", "POST", "PUT"], # Methods for retry raise_on_status=False # Don't raise error after final retry ) # Create an HTTP adapter with the retry strategy adapter = HTTPAdapter(max_retries=retry_strategy) # Create a session and mount the adapter http = requests.Session() http.mount("https://", adapter) http.mount("http://", adapter) # Update session headers with API token http.headers.update( {"Content-Type": "application/json", "Authorization": f"Bearer {api_key}"} ) return HubSpot(access_token=api_key, http=http) # Create HubSpot API clients for different calls api_client_companies = create_client(get_access_token("HUBSPOT_COMPANIES", "companies")) api_client_contacts = create_client(get_access_token("HUBSPOT_CONTACTS", "contacts"))

Could anyone share insights or suggest further improvements? Thank you!

Re: API Contacts: Remote end closed connection without response

RubenBurdin — Mon, 14 Oct 2024 18:58:52 GMT

Hey there,

Founder at Stacksync here. RemoteDisconnected is a common HubSpot error that can happen for a variety of reasons, coming due to HubSpot issues (most often) or sometimes network issues depending on how reliable is your cloud provider and network tier.

At Stacksync, we have several parallel data read/write processed for a single HubSpot instance. We have implemented a retry module on the API call with exponential backoff delays. It ensures that you do not exceed rate limits, as well as being robust to repeated RemoteDisconnected errors (the error can happen sometimes 1-4 times in a row, and this can lead to coordination issues between processes if you parallellize queries).

For context, Stacksync powers real-time and bidirectional sync between HubSpot and Databases like Postgres, Snowflake, BigQuery, MySQL and more.

Re: API Contacts: Remote end closed connection without response

GAlinio — Mon, 14 Oct 2024 21:01:17 GMT

Hi @RubenBurdin,

Thank you for your response about the RemoteDisconnected error in HubSpot. I appreciate the information on how Stacksync handles parallel data processing and the retry module. It’s very helpful for improving my script. I have a few quick questions:

Can you share more details about how you implemented the retry module? Any best practices or specific libraries you recommend?
Could you provide examples of how you manage rate limits effectively in a high-volume environment?

I look forward to using these strategies in my setup.

Thanks again!
Gerald

Re: API Contacts: Remote end closed connection without response

RubenBurdin — Wed, 16 Oct 2024 21:46:42 GMT

Hey @GAlinio,

Unsure I fully understand the question here. Would you like to see code or architecture?

Happy to chat about it at https://cal.com/rubenburdin/20min (feel free to book anytime) if you wanna deep dive together into the topic.

Cheers,

Re: API Contacts: Remote end closed connection without response

RubenBurdin — Mon, 03 Feb 2025 18:31:11 GMT

This article solved it for me https://www.stacksync.com/blog/explaining-hubspots-remotedisconnected-error

Re: API Contacts: Remote end closed connection without response

RubenBurdin — Tue, 01 Apr 2025 14:45:39 GMT

I had the same issue when building our syncs. In custom-code, we had to implement retries (ideally retry 4 times at intervals of 5 seconds). It is possible that the issue still persists after 4 retries altough most case should pass with this simple rule.

Since our data was massive and we had to iterate over 200k+ Associations too, we just ended up using Stacksync managed two-way sync with our database (postgres) and it has never failed so far. They manage this issue internally.