Contacts delayed in API calls

[Avanti_Ketkar]

Hello,

We run a daily job to call HubSpot API and get the contact data in out data warehouse. We are noticing that some of the contacts do not appear in the data for several days after they are created.

Here are some examples:
marci@ziplostpines.com - create date is 5/21/2018 but this came in our system only on 5/31

bbridgewater1989@gmail.com
and
terri@trustmelaw.com: Both came in more than 2 days later.

We had 11 of these coming in late last week.

Here are some details around the API calls:

• When did you set up this API? How did you set up this API?
  *** We started using HubSpot about 3 years ago and use the URL https://api.hubapi.com. We use the API key that starts with '99722bf9'.

• Are you using any of the integration to do this?
  *** No we are not. We are making direct API calls from out Python code.

Which AIP are you using ( this one? )
*** https://developers.hubspot.com/docs/methods/contacts/get_recently_updated_contacts

Are you using developer portal to do this? If so could you provide me the developer portal ID?
** We use this on Production only.

Can you please take a look and let me know why this would happen? Is there a way to avoid this from happening in the future?

Thanks,
Avanti

[Derek_Gervais]

Hi @Avanti_Ketkar,

Just to make sure that we're on the same page, how exactly are you pulling these contacts? Are you paging backwards in time until you reach the previous day? A contact created in HubSpot should appear in the results of that endpoint almost immediately; is it possible that you're not paging far enough back in time to pick up these contacts, so they only appear a day or two later when they've been updated again?

Can you send me your Hub ID so that I can take a look at your portal? It'd also be convenient if you had some example contacts that were present in HubSpot but were not appearing in the response from the 'Get recently updated contacts' endpoint.

[Avanti_Ketkar]

Hi,

The HubID is 442215. The examples are already in my post. Those contacts did not appear for 2 days.

We make the direct API calls from the PHP code. We do not limit the number of records in the call or use paging.

Thanks,
Avanti

[Derek_Gervais]

Hi @Avanti_Ketkar,

I'm not entirely clear on your process; can we clarify a couple things to make sure that we're on the same page?

• "We do not limit the number of records in the call..."
  • Are you saying that you're not including a &count= parameter in your request URL? If you're not explicitly including a &count= parameter, then by default that endpoint is only returning 20 contacts, since that's the default
• "[We do not] use paging."
  • Does this mean you're only making a single request, and not making subsequent requests including the vidOffset and timeOffsetparameters?

Before we dig in deeper, it's important to note that this endpoint does not return all recently modified contacts in a single response. It returns a single batch of contacts (20 by default, or equal to the &count= parameter) and then two fields that you can use to get the 'next' batch (vid-offset and time-offset).

Can you clarify whether or not this is part of your process? If you're not doing any paging at all, then it's likely that the 'missing' contacts just aren't included in the batch of 20 you're requesting, and it's not until a later update that they end up being included in the response.

[Avanti_Ketkar]

We're setting &count=100 to fetch 100 records at a time (the max that HubSpot allows you to fetch in a single call).

We're also doing paging using both the &vidOffset= and &timeOffset= params. We keep fetching page after page of results until the has-more field is false.

The records that do not show up on a particular day after we have exhausted the calls, show up a few days later.

Thanks,
Avanti

[Derek_Gervais]

Hi @Avanti_Ketkar,

Apologies for the delay here. I haven't been able to reproduce this issue on my end, and based on the information you've given me here it sounds like you're doing everything right. In order to really dig into what's going on, I'd need to see a contact record that exists in your portal, but is not appearing in the results returned by the 'Get recently modified contacts' endpoint. I can't use an older example, since it the record is in HubSpot and also appears in the response returned by the 'Get recently modified contacts' endpoint, the record will no longer be demonstrating the incorrect behavior.

Do you have / can you create a record in this state so that I can take a closer look? If you're not seeing records in this state, is it possible that the records in question are being returned in the response from the 'Get recently modified contacts' endpoint, but are being lost in translation somewhere between there and you system?