I am working on an ELT tool, using APIs I would like to know how can i make sure that I get only updated data(new data) in second time API call, something like offset we use for single API call to get all the data, as per the reference doc this Offset is not long live. Or is there a fixed time interval before which I can use the offset from first call and make the second request
The offsets provided accross all of our paginated APIs are not long-lived; while the exact lifespan can differ depending on the API, none of them should be considered valid on a larger timescale than an hour.
Depending on what specific data you're pulling from HubSpot, you might consider using some of the recently modified / recently created endpoints (e.g. Recently Created Contacts). While you shouldn't store the offset value, you can store the addedAt timestamp for the last contact you've pulled. The next time you go to pull recently created contacts, you can stop once you've paged back far enough to reach addedAt values that you've already seen.
I see; thanks for including an example! The email events API doesn't support a `created` parameter: Instead, you can use a combination of `startTimestamp` and `endTimestamp` to define the timeframe for which you want to pull data, and then paginate through the data using the `offset` values provided in each response.
The offsets provided accross all of our paginated APIs are not long-lived; while the exact lifespan can differ depending on the API, none of them should be considered valid on a larger timescale than an hour.
Depending on what specific data you're pulling from HubSpot, you might consider using some of the recently modified / recently created endpoints (e.g. Recently Created Contacts). While you shouldn't store the offset value, you can store the addedAt timestamp for the last contact you've pulled. The next time you go to pull recently created contacts, you can stop once you've paged back far enough to reach addedAt values that you've already seen.
I see; thanks for including an example! The email events API doesn't support a `created` parameter: Instead, you can use a combination of `startTimestamp` and `endTimestamp` to define the timeframe for which you want to pull data, and then paginate through the data using the `offset` values provided in each response.
.png "Alunos da HubSpot"){kind=icon}
