topic Re: am unable to filter response.status_code == 400 for campaigns data in APIs & Integrations

am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 07:28:00 GMT

am unable to put response.status_code == 400 if offset value is not present [error : ]and continue with the offset values which has data for the campaigns below is the code am using .. need help

has_more = True
offset = ""
offset_str= str(offset)
limit = 10

# List to store the data
offset_list=[]
data_list = []
campaign_ids_list=[]
prev_campaign_ids=set()
loopiteration=1
while has_more:
url = "https://api.hubapi.com/email/public/v1/campaigns?limit=10&offset="+offset
print("loopiteration :",{loopiteration})
loopiteration +=1
print("samsssss",url)
#print(url+offset)
response = requests.request("GET", url, headers=headers)
if response.status_code ==200:
#json_data= response.json()
json_data= json.loads(response.text)
print(json_data)
campaigns =json_data.get('campaigns',[])
campaign_ids=set([campaign.get('id') for campaign in campaigns])
campaign_ids_list.extend(list(campaign_ids))
#campaign_ids_list.extend(campaign_ids)
campaign_count= len(campaign_ids)
print("campaign_ids:",campaign_ids)
print("length of prev_campaign_ids:",len(prev_campaign_ids))
#offset +=limit
# Check if there are more records
has_more = json_data.get("hasMore", False)
print("has_more",has_more)
if has_more:
offset = json_data.get("offset", str(''))
offset_list.append(offset)
print("offf",offset_list)
else:
#offset =
has_more=False

else:
print(f"Request failed with status code {response.status_code}")
has_more=False

Re: am unable to filter response.status_code == 400 for campaigns data

himanshurauthan — Wed, 20 Sep 2023 08:21:37 GMT

Hello @9kumar

The issue in your code is that you are not checking for the presence of the offset key in the JSON response before adding it to the offset_list. This can cause an error if the response does not contain the offset key, such as when the offset value is not present.

You can use this code

offset_list = [] has_more = True offset = "" offset_str= str(offset) limit = 10 # List to store the data data_list = [] campaign_ids_list=[] prev_campaign_ids=set() loopiteration=1 while has_more: url = "https://api.hubapi.com/email/public/v1/campaigns?limit=10&offset="+offset print("loopiteration :",{loopiteration}) loopiteration +=1 print("samsssss",url) #print(url+offset) response = requests.request("GET", url, headers=headers) if response.status_code ==200: #json_data= response.json() json_data= json.loads(response.text) print(json_data) campaigns =json_data.get('campaigns',[]) campaign_ids=set([campaign.get('id') for campaign in campaigns]) campaign_ids_list.extend(list(campaign_ids)) #campaign_ids_list.extend(campaign_ids) campaign_count= len(campaign_ids) print("campaign_ids:",campaign_ids) print("length of prev_campaign_ids:",len(prev_campaign_ids)) # Check if the `offset` key is present in the JSON response if "offset" in json_data: offset = json_data.get("offset", str('')) offset_list.append(offset) else: # The `offset` key is not present in the JSON response, so stop iterating has_more = False print("offf",offset_list) else: print(f"Request failed with status code {response.status_code}") has_more=False

by this code you can check for the presence of the offset key in the JSON response before adding it to the offset_list. If the offset key is not present, the code will stop iterating.

Re: am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 08:29:41 GMT

Thanks @himanshurauthan for swift reply

actually am able to get 4 requests in a loop but last request am getting error

Request failed with status code 400

so i would like to skip the response request which is failing or response.status==400 and do iteration for response==200 but loop is ending here itself because i have given has_more=True . and offset is in this format

AQAAAYqw+Xr4AAAAAEEUh7Q=

need help here how to ignore failed response ==400 and get all response=200

Re: am unable to filter response.status_code == 400 for campaigns data

himanshurauthan — Wed, 20 Sep 2023 08:35:25 GMT

ok this is the modified code

import requests import json # Initialize variables offset = "" limit = 10 data_list = [] campaign_ids_list = set() # Use a set to ensure unique campaign IDs while True: # Infinite loop to handle skipping requests with status code 400 url = f"https://api.hubapi.com/email/public/v1/campaigns?limit={limit}&offset={offset}" response = requests.get(url, headers=headers) if response.status_code == 200: json_data = json.loads(response.text) campaigns = json_data.get('campaigns', []) campaign_ids = set([campaign.get('id') for campaign in campaigns]) campaign_ids_list.update(campaign_ids) # Add campaign IDs to the set print(f"Fetched {len(campaign_ids)} campaigns. Total unique campaigns: {len(campaign_ids_list)}") # Check if there are more records has_more = json_data.get("hasMore", False) if has_more: offset = json_data.get("offset", "") else: break # Exit the loop when there are no more records elif response.status_code == 400: print(f"Received status code 400. Skipping offset: {offset}") # If you want to continue with the next offset, you can increment it here # For example, increment offset by 1: # offset = str(int(offset) + 1) # Make sure to handle offset conversion and overflow as needed. else: print(f"Request failed with status code {response.status_code}") break # Exit the loop on any other status code # Now, you can continue processing the campaign_ids_list or perform any other desired actions.

I've modified the loop to run indefinitely using the condition "while True." This alteration ensures that the loop will persistently process successive requests, even if it encounters a response with a 400 status code. Your ability to adapt the logic within the "elif" block to manage offset increments remains intact. The loop will terminate when it comes across a status code different from 200 or 400, signifying the end of processing.

Re: am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 08:49:06 GMT

The above code is working upto 13 iterations than again I got 400 error and loop was stopped

Re: am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 08:52:32 GMT

@himanshurauthan thanks for sharing code .

For the shared above code it has taken only

Fetched 10 campaigns. Total unique campaigns: 10
Fetched 10 campaigns. Total unique campaigns: 20
Fetched 10 campaigns. Total unique campaigns: 30
Fetched 10 campaigns. Total unique campaigns: 40
Fetched 10 campaigns. Total unique campaigns: 50
Fetched 10 campaigns. Total unique campaigns: 60
Fetched 10 campaigns. Total unique campaigns: 70
Fetched 10 campaigns. Total unique campaigns: 80
Fetched 10 campaigns. Total unique campaigns: 90
Fetched 10 campaigns. Total unique campaigns: 100
Fetched 10 campaigns. Total unique campaigns: 110
Fetched 10 campaigns. Total unique campaigns: 120
Fetched 10 campaigns. Total unique campaigns: 130
Received status code 400. Skipping offset: AQAAAYqvLhh+E+yM=
Received status code 400. Skipping offset: AQAAAYqvLhh+E+yM=
Received status code 400. Skipping offset: AQAAAYqvLhhE+yM=
Received status code 400. Skipping offset: AQAAAYqvLhh+E+yM=
Received status code 400. Skipping offset: AQAAAYqvLhh+E+yM=

for status 400 it is repeating same offset value not going further

Re: am unable to filter response.status_code == 400 for campaigns data

himanshurauthan — Wed, 20 Sep 2023 09:08:25 GMT

I think your code is repeating the same offset value for status 400 is because maybe you are not checking if the offset value is already in the offset_list. If the offset value is already in the offset_list, then your code will skip the current iteration of the loop and move on to the next iteration.

Re: am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 09:19:00 GMT

@himanshurauthan oh can you help me if you dont mind how to check and get into next iteration please am very new to this

Re: am unable to filter response.status_code == 400 for campaigns data

9kumar — Wed, 20 Sep 2023 10:38:52 GMT

Hi @himanshurauthan sir , are you there ? can you please help me i got stuck I tried