Contacts Query by Recently Modified: Not All Results Returned
Jun 28, 2020 8:02 PM
With regards to this endpoint:
In some situations we use this to collect details of Contacts created / modified since last last time a full sync ran. The logic used is typically:
- Note the time of this sync (START_TIME)
- Look up the time of the last successful sync (LAST_START_TIME i.e. START_TIME of last sync that ran to completion)
- Use the above endpoint to query for all Contacts created / modified since LAST_START_TIME minus 5 minutes - the 5 minutes allows for potential time skew between the server running the integration and HubSpot servers
- Save START_TIME from (1) to use as the starting time for the next sync
Occasionally we find that some Contact record changes are not detected by the sync - we can see that a Contact has been created/modified however the above query "missed" passing this Contact to the sync process. It is safe to assume that the server that runs the integration has an accurate PC clock (i.e. START_TIME will be accurate to +/- 1 second).
- Is there a flaw in the logic as outlined above?
- This seems to happen in larger HubSpot accounts during times when a large number of data changes are happening, e.g. importing of thousands of Contacts that trigger lots of list / workflow update. Consequently we are wondering if HubSpot's internal engine may e.g. start processing a batch at START_TIME minus 1 minute - lets say it takes HubSpot 10 minutes to process that batch of changes - and then, at START_TIME + 9 minutes, HubSpot commit's ALL changed Contacts with some of them having a last-modified timestamp that will now be less that the 5 minute window allowed for above. So: can HubSpot sometimes apply changes to a Contact and set the last-modified timestamp of that change to a time that is more than 5 minutes behind "current" wall clock time?