API - Get recently modified contacts returning no result when same contact modified (has-more: true, vid-offset:0) after pull
Please validate if this may be an issue , business case, pulling 1000 contacts in batch(2 at a time) and update same contact’s properties in each batch. When doing so in loop (after 1st, 2nd batch ), the vid-offset/time-offset get out of sync and starts returning zero results.
Call API /recently_updated/contacts/recent?count=2&vidOffset:x&timeOffset:a
Returns next 2 recent with ( has-more : true; vid-offset : y , time-offset: b )
Call API to update contact property for these 2 contact (example : jobtitle)
Call API /recently_updated/contacts/recent?count=2&vidOffset:y&timeOffset:b
Returns with ( has more : true; vid-offset : 0 , time-offset:0 )
NOTE if we change the above URL ( increase count to 7 ) and make the call again with same vidoffset and timeoffset
/recently_updated/contacts/recent?count=7&vidOffset:y&timeOffset:b it return 3 records only with ( has-more : true; vid-offset : z , time-offset:c )
Please validate/ look into it …
We will be implementing a wordaround for now, but this behavior is not documented or it is an issue
Just wanted to reach out with a quick update. I’m still working with product to get some precise information on how the system is expected to work in this scenario. I appreciate the patience here, there are some complexities on our end that make this difficult to troubleshoot. I’ll continue digging into this and reach out next week with some more info.
API - Get recently modified contacts returning no result when same contact modified (has-more: true, vid-offset:0) after pull
Thanks @przemyslaw.celej, Will keep that in mind. The delay does not help when using the vid-offset and time-offset together (even waiting for a day)…but based on your input, will try the delay only for time-offset based on the process explained below.
@Derek, thanks for continuing to look into this as a priority… in regards to your question -
Yes, we are using both vid-Offset and time-Offset … But when the process encounters the above scenario, it uses only the time-offset at that point to make the next call,it then succeeds returning a new set of vid-offset and time-offset (by passing the updated records), the process then goes back to using that new vid-offset and time-offset (which has a time-offset less than the one requested for, again hoping 100 plus records were not updated at the same time-offset) . It seems like a potential risk, but based on couple rounds of tests, it has worked for us…
( Based on above suggestion from @przemyslaw.celej, we may add a delay, in case ‘time-offset only’ call fails, but have not encountered that as yet )
We are in midst of processing xxx,xxxK records, will keep posted if this process fails … but hope to get a firmer solution on this from Hubspot.
API - Get recently modified contacts returning no result when same contact modified (has-more: true, vid-offset:0) after pull
Hey all,
Still digging into this, but I wanted to check in. Are you all using bothvidOffsetandtimeOffset? Both are necessary to accurately page backward through lists of contacts (recently modified, recently updated, etc.).
The workaround is to wait few (or more) seconds and retry the call. It looks that, if some contact is modified while calling to “get recently updated” the call returns it’s contact’s lastmodifieddate (or just largest lastmodifieddate)
I’m currently digging into this with my team, since I’m not clear on the expect behavior of this endpoint in all cases. I’ll post back here with an update once I have an update on this behavior.
Just to make sure that we’re on the same page, your current process goes like this:
Pull 2 contacts from /contacts/v1/lists/recently_updated/contacts/recent
Update those contacts
Pull the next 2 contacts from /contacts/v1/lists/recently_updated/contacts/recent
Update those next 2 contacts
etc.
Is that correct? I would suspect that updating contacts before pulling subsequent pages from the recently modified endpoint could invalidate the offset values, since the new updates are changing the log of recently modified contacts. I’ll have to conduct some more testing to fully understand the behavior here, but my gut tells me that your best bet would be to try and request the contacts you need up front and then update them, if possible.
API - Get recently modified contacts returning no result when same contact modified (has-more: true, vid-offset:0) after pull
Hi Derek, Appreciate you looking into this. You have the steps correct. Also your suspicion that offset values get invalidated is correct. We have an ongoing process (batch) pull, and a real time push happening into Hubspot at same time (opt out, transactional email, other contact property updates from external CRM ) so hard to circumvent preventing an update during the pull.
Offsets getting invalidated defeats the concept of doing the next pull based on offsets without accurate result.
Work around, is to just use the time-offset in case of failures but that is an extra step with possibility of pulling some same data again. We have tested a bit and counting on this solution to work else this issue is a bigger priority and also should be validated/documented/resolved quickly (if findings are true).
.png "HubSpot Alumni"){kind=icon}
