Campaigns, Events API request offset behavior.

[Bulat]

The question concerns these two endpoints:

https://api.hubapi.com/email/public/v1/campaigns (get campaign IDs with recent activity for a portal)

https://api.hubapi.com/email/public/v1/events (get email events)

I'm trying to invent an algorithm that will let me incrimentally synchronize events with another system. The one that will do this in a robust and efficient way.

The questions:

1) For how long offsets returned by the API stay relevant? 1 day? 1hour? 15 min? 1 min? Other?

2) What will happen if a campaign activity happens (campaings gets updated thus changing the order of campaigns) while I'm going through campaigns list pages? Will offset returned by the API be still actual/relevant? Will the results acquired using last offset be returned in the original order? Or will the order change according to recent campaign activity updates?

Thanks in advance!

[WendyGoh]

Hey @Bulat,

As mentioned by Isaac on this similar forum thread: https://community.hubspot.com/t5/APIs-Integrations/Email-Events-API-not-returning-full-list-of-event..., 

The same offset value could be used within a short window.

That said, the endpoint is intended to be read in reverse-chronological order (newest events will come first). Said differently, you should start at the beginning and page backwards for each job rather than store the offset to start with next time. 

Hope this helps to shed some lights!

[Bulat]

Hi! Thanks for the reply. That topic does not answer my question.

What comes closer to my is this one: Email Events Bulk Dump 

The offset is idempotent in that you can repeatedly query the same offset value again and it will return the same events, regardless if more recent events have occurred. That said, I've only tested this within a twenty minute window. Our documentation does state that the offset value "is not meant to be long-lived," so I'm not sure how long the value can be used.

And for me it is this two things that matter:

1) For what period of time is offset idempotent? Is its idempotence guaranteed for atleast a second, minute, 15 minutes, more? It is crucial to know this for deriving an algorithm of a bulk retrieval (even one based on the idea of eventual consistency). So, if temporal parameters of this idempotence are not declared/defined/guaranteed then one can not consider it as idempotent at all and has to consider only the first page to be retrievable error-free. Is there anybody who can shed a light on this?

2) Will server inform me that an offset has expired or a response became not idempotent? If yes - how it will do that? Some particular HTTP error? JSON?

I need this for those two endpoints I've mentioned earlier.

[WendyGoh]

Hey @Bulat,

Thanks for the further clarification. 

I'll check in with our product team to see (1) if the offset value on these two endpoints: Get email events | Email Events API and Get campaign IDs sorted by recent activity for a portal | Email Events API expired. (2) If it does expired, how are we informing it through the API request. 

I'll keep you posted!

[Bulat]

Thanks.

There's one thing - for #1 it is not 'is it expired', it is:

Is the response idempotent for a specified period of time? And is it guaranteed for that period of idempotence to be at least 15 min (1min, 1 hour, etc.)?

[WendyGoh]

Hey @Bulat,

Here's an update after checking in with our Product team:

Currently, we have no guarantees on idempotency or any expiration policy in place. As stated on the Get email events endpoint:

The returned offset is an opaque token used internally to keep track of the position in the result set. It's not meant to be used for anything other than subsequent requests on this endpoint with the same query parameters (excluding offset itself and limit). Further, it's meant to be short-lived.

This works the same for endpoint - Get campaign IDs with recent activity for a portal.

I do agree that this is an interesting use case and moving forward, if you haven't already, I'd recommend jumping over to the HubSpot Ideas Forum and posting this idea there so that it's visible to the HubSpot product team and other HubSpot customers and developers. Hopefully this will gather more interest.