API Request Limit

[SM6]

Hi, I am trying to get a list of all of my contacts from hubspot. I have about 800,000 contacts and a starter account, which means I should have 250,000 possible requests and my code should take 8,000 requests. I have checked that each request gets 100 contacts each time. However, after running it twice today, it says that I have already hit the limit:

You've succesfully parsed through 171200 contact records and added them to a list 
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
`--NotebookApp.iopub_data_rate_limit`.

Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)

This is what my code looks like:

import hubspot
from pprint import pprint
from hubspot.crm.contacts import ApiException
import urllib
import requests
import json

max_results = 500 
hapikey = 'XXX' 
count = 100
contact_list = []
property_list = []
get_all_contacts_url = "https://api.hubapi.com/contacts/v1/lists/all/contacts/all?"
parameter_dict = {'hapikey': hapikey, 'count': count}
headers = {}

# Paginate your request using offset
has_more = True
while has_more:
    parameters = urllib.parse.urlencode(parameter_dict)
    get_url = get_all_contacts_url + parameters
    r = requests.get(url=get_url, headers=headers)
    try:
        response_dict = json.loads(r.text)
        try: 
            has_more = response_dict['has-more']
            contact_list.extend(response_dict['contacts'])
            parameter_dict['vidOffset']= response_dict['vid-offset']
            print('loop finished')
        except:
            break    
    except:
        continue

list_length = len(contact_list) 

print("You've succesfully parsed through {} contact records and added them to a list".format(list_length))
print(contact_list)

Does anyone know why this is happening? Thank you in advance!

[JBeatty · Accepted solution]

Hi @SM6,

That error response you posted "IOPub data rate exceeded", Is a Jupyter Notebook error due to trying to print the whole contact_list, not the actual request error. If you could catch the error that HubSpot is returning and log it, so we could see, that would be helpful.

Without seeing the error itself my assumption is that you had reached the "Burst: 100/10 seconds" limit, because I do not see any rate-limiting with your loop, though I will admit my python is a little rusty, so I might just be misreading your code.

Another suggestion I do have is depending on what you are doing with this data, it might be better to perform a manual export. Following this article: Export your records, will let you export all of your data in csv format, which might be an easier way for you to get that data that you need.

[SM6]

Hi, thank you for your response! It turns out the error was caused by the print statements. Once I deleted them, the code works as intended.