To periodically extract contact subscription preferences, ideally on a daily or weekly basis, we face the challenge of batch processing limitations, which allow only 100 contacts at a time. For millions of contacts, this results in 10,000 calls daily. We aim to significantly reduce this number.
Is there a way to extract this data incrementally? Specifically, can we perform a full load initially and then, on subsequent days, retrieve only the subscription preferences that have changed?
Other ideas to impove this process are also very welcome!
Hey, @AD18 Thanks for the great question. Looking at the limitation documentation, we care not able to get around the batch size limitation. My initial thoughts went to the Search or Webhooks API. But didn’t find a great match. This post from @zach_threadint offers one workaround you might investigate to see if it’s adaptable for your use case, using Workflows and Webhooks in Workflows — Webhook triggered when a contact unsubscribes from email. And we’ll definitely leave this open in case other community members have any feedback for you.
Unfortunately, there isn’t a built-in way to incrementally load only the changed subscription preferences. The batch size limit means that for large contact lists, pulling all preferences daily isn’t practical.
The main workaround is to use HubSpot Workflows and Webhooks. For example, you can set up a workflow to trigger a webhook whenever a contact updates their subscription preferences (like unsubscribes). This lets you capture changes as they happen, so you don’t need to pull everything each day.
If you need to track all types of changes (not just unsubscribes), you might need to store a local copy of your subscription data and compare it with new pulls, but this still requires fetching all data unless you have a way to log every change.
It’s not perfect, but using webhooks is usually the most efficient option available right now. If you want more details on setting this up, let me know!
Thanks for the great question. Looking at the limitation documentation, we care not able to get around the 
— Jaycee