Hubspot Python API Retry

Developer Support APIs & Integrations
1

Hello! I have implemented the Retry based on the Hubspot API Python GitHub repo and urllib3 on an AWS lambda, but doesn’t seem the retry for failed attempts is happening. This is the setup:

retry = Retry(

total=3,

backoff_factor=0.7,

allowed_methods=False,

status_forcelist=(429, 500, 502, 504, 400, 404),

)

access_token = os.environ[‘HUBSPOT_API_TOKEN’]

api_client = HubSpot(retry=retry)

So I expect that when I call this, there will be three retries:

api_response = api_client.crm.contacts.search_api.do_search(public_object_search_request=public_object_search_request)

But I don’t see any.
Could you please help me to fix/debug this?

2

Hi, @TonyT :waving_hand: Thanks for reaching out! I’d like to invite some of our community members to join the conversation — hey @tominal @taran42 @ChrisoKlepke, do you have any tips you can share with @TonyT?

Thank you very much for taking a look! — Jaycee

3

Thank you for tagging @Jaycee_Lewis

Hey @TonyT ,

it looks like the creation of the API client might be a bit wrong. Can you try:

api_client = HubSpot(access_token=os.getenv("access_token"), retry=retry)

You probably did that already, but make sure to import the retry library at the top of your code:

from urllib3.util.retry import Retry

The call of the retry function looks correct to me, and also worked in my tests with the modifications.

Maybe this helps.

If you found this post helpful, consider helping others in the community to find answers faster by marking this as a solution. I’d really appreciate it.

Cheers,

Chriso

4

Hey @ChrisoKlepke ! Thank you for the prompt answer. No that part was fine I didn’t paste the whole code by accident, this is how it is (and except retry it works well, plus its also as in the documention in the Python Github: GitHub - HubSpot/hubspot-api-python: HubSpot API Python Client Libraries for V3 version of the API · GitHub)

import os

import json

import time

from hubspot import HubSpot

from typing import Optional

from urllib3.util.retry import Retry

from hubspot.crm.contacts import SimplePublicObjectInput, PublicObjectSearchRequest, ApiException

retry = Retry(

total=3,

backoff_factor=0.4,

allowed_methods=False,

status_forcelist=(429, 500, 502, 504, 400, 404),

)

access_token = os.environ[‘HUBSPOT_API_TOKEN’]

api_client = HubSpot(retry=retry)

api_client.access_token = access_token

so when I call

api_response = api_client.crm.contacts.search_api.do_search(public_object_search_request=public_object_search_request)

and there is no Contact I get back this value (converting it to Dict) and I don’t see any retries happening:

{‘paging’: None, ‘results’: [], ‘total’: 0}

Shouldn’t be there a Retry in this case? How could I verify if the retry happens?
Also thanks @Jaycee_Lewis

5

I see, Tony.

I think I can help you out a little more here, mate.

It seems like the call you’re making against the search API doesn’t return any errors. It looks like that the search request doesn’t align with any value from your contacts.

If that is really the case, which I highly suspect, the server’s response code is 200. The status codes you’re looking for in your retries are 429, 500, 502, 504, 400, 404. Which makes sense, as you would usually only catch errors really.

If you use the do_search_with_http_info method instead of the normal do_search, you get that information as well. It should look something like this:

api_response = api_client.crm.contacts.search_api.do_search_with_http_info(public_object_search_request=public_object_search_request)

For example, this is what I’m getting back:

({‘paging’: None, ‘results’: [], ‘total’: 0}, 200, HTTPHeaderDict({‘Date’: ‘Fri, 17 Mar 2023 14:50:59 GMT’, ‘Content-Type’: ‘application/json;charset=utf-8’, ‘Content-Length’: ‘24’, ‘Connection’: ‘keep-alive’, ‘CF-Ray’: ‘7a960d289b12904e-FRA’, ‘Strict-Transport-Security’: ‘max-age=31536000; includeSubDomains; preload’, ‘Vary’: ‘origin, Accept-Encoding’, ‘CF-Cache-Status’: ‘DYNAMIC’, ‘Access-Control-Allow-Credentials’: ‘false’, ‘X-HubSpot-Correlation-Id’: ‘11533ab1-d998-419f-a714-5cf44e65b54d’, ‘X-HubSpot-RateLimit-Daily’: ‘500000’, ‘X-HubSpot-RateLimit-Daily-Remaining’: ‘499977’, ‘X-Trace’: ‘2BCF7D4EBC5BFB6512CAAD0A23663831ADBD89F898000000000000000000’, ‘Report-To’: ‘{“endpoints”:[{“url”:“https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=wHhEhNZ2BERa%2BgG%2BiPQO2eE4XyELJd3xn0Cg%2BmekEY%2Bx%2BCLx3Xygzdf4OTJJoINF%2BZgnc9XOp3PKsTJNH1v4vbf39octrIlyplxnvz2l%2FkgWjrrD9Lg4dPuVrtPejbjnMJZifBz%2FxZCPJIdF”}],“group”:“cf-nel”,“max_age”:604800}’, ‘NEL’: ‘{“success_fraction”:0.01,“report_to”:“cf-nel”,“max_age”:604800}’, ‘Server’: ‘cloudflare’, ‘alt-svc’: ‘h3=“:443”; ma=86400, h3-29=“:443”; ma=86400’}))

Did you manage to find any contact with some other filter combination before?

You would see a lot of error messages in the response if the retries would fire. Also it says something similar to:

Max retries exceeded with url: /crm/v3/objects/deals/search (Caused by ResponseError(‘too many 400 error responses’))

Cheers,

Chriso

6

@ChrisoKlepke Awesome, thanks for the cool detailed answer and also for letting me know what a retry would look like in the logs. That makes sense if the returned value is 200.
But also if the search returns not found shouldn’t the proper response would be 404 not found instead of 200? Is there a way to make the retry happen when there is no found contact with this 200 response? I wonder, how to trigger that retry.
Usually there is no issue finding the contacts, only if we try to find the contact shortly after they signed up via a Hubspot form and we try to query the Contact before its created. A follow up question would be, usually how long does it take to create a new contact after form submission? Thanks again!

7

Hey @ChrisoKlepke ! Here is the problem, I have tried that with info method and when I get back an empty result array I get back a 200 response so there is no retry. It should be a 00 not found, how can I manually trigger the retry could you give some advice? I could say if the results array is empty retry, but ideally I think the response shouldn’t be 200 either.

({'paging': None, 'results': [], 'total': 0}, 200, HTTPHeaderDict({'Date': 'Wed, 05 Apr 2023 13:16:03 GMT', 'Content-Type': 'application/json;charset=utf-8', 'Content-Length': '24', 'Connection': 'keep-alive', 'CF-Ray': '7b321038fd32695b-FRA', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'Vary': 'origin, Accept-Encoding', 'CF-Cache-Status': 'DYNAMIC', 'Access-Control-Allow-Credentials': 'false', 'X-HubSpot-Correlation-Id': '16b7cd3b-bdd3-42dd-ae1a-723015ba1f9d', 'X-HubSpot-RateLimit-Daily': '500000', 'X-HubSpot-RateLimit-Daily-Remaining': '490013', 'X-Trace': '2BAA145C0E1DE914DF3EEBC1C64B3328474B9BAEC8000000000000000000', 'Report-To': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=vUZ%2FTO4DtDnFbWPBYboEHpNE3gsH3K9UWiBTZ%2BU1Rh2by9xnr%2FjPJtit6jV6GKaU8GmP6fxhp%2BfdeuWhJRtC2urWMWqUeeERblGVKi8Ovr039KAUx5fpVWE0qBZYP68T"}],"group":"cf-nel","max_age":604800}', 'NEL': '{"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}', 'Server': 'cloudflare', 'alt-svc': 'h3=":443"; ma=86400, h3-29=":443"; ma=86400'}))
8

I have managed to write a custom retry function, because the 200 response doesn’t allow me to use the retry of the recommended library. Maybe you could change the response to 400 if there are no results? Other than that thank you for the help!