API Contacts: Remote end closed connection without response

[camera]

Hello people!

I'm doing some requests to the Contacts endpoint to get all of my contacts. But, sometimes I get an error that I can't handle. I think that the server isn't responding to my request.

There are a lot of contacts (700k) and I'm running the code below in a for iteration.

I know about the rate limits, and I'm sure I'm not passing thru.

url = 'https://api.hubapi.com/crm/v3/objects/contacts'

headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + <token>
}

params = {
'limit': 100,
'properties': <list of properties>,
'after': <next_paging number>
}

response = requests.get(url=url, headers=headers, params=params)

Sometimes I get the error below, sometimes I don't. 

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

http.client.RemoteDisconnected: Remote end closed connection without response

urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

I've summarized the error so it doesn't get too big.

Can anyone help me, please?

[SBurwash]

I came accross this issue also; would there be a way to solve it? Our syncs take a long time since we need to iterate through all of our associations, and this is making our ETL tool fail

[RubenB]

I finally used a managed service for real-time and two-way sync to solve the error: https://www.stacksync.cloud/

It works for me now.

[dennisedson]

@camera 

From my searches, this appears to be a python bug perhaps 🤔

@JBeatty , @BJacobson  either of you run into this before?

[camera]

Hi @dennisedson !

Thanks for reply! It's not a python bug. It's about the non-response of some messages by Hubspot's API server.

I made some improvements in the API request in the python script. I configured the retry factor from the urllib3 library (https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#module-urllib3.util.retry) and now my pipeline it's not crashing anymore.

But, even so, I would like to know why the Hubspot API server is not answer some of my requests...

Thanks in advance.