Contact Property Batch Update via Webhook

[mainespotter]

I apologize in advance if this post gets kind of long 😉

We have a webhook that is triggered by contact create or contact zip property change.  We make a call to HubSpot to get some data on the contact, when the data comes back from HubSpot we use it to do a database lookup on our end and then send some data back to HubSpot to populate a custom contact property we we created (let's call the property 'custom_1').  This is all working great.

We needed to backfill the custom_1 property, and came up with a way to have that same webhook be triggered en_masse.   That is also working great... almost.

I say "almost" because what does NOT seem to be working great is how HubSpot is handling the en_masse calls:

Issue #1: We're seeing a lot of "Concurrent timeout" with "Webhook attempt timed out due to a backlog of concurrent requests." errors.   A call to the settings URL for the app shows that max_concurrent_requests is 6.  Perhaps that's too low?  We've used the UI to create the webhooks, and I don't see anyway to change max_concurrent_requests anywhere.  In case it's relevant, we have Event Throttling set to 6 per minute and, to be honest, I have no idea if that's high or low or has anything to do with this issue

Issue #2: It seems as if HubSpot is making the en_masse calls hits to our webhook sporadically... at least the Monitoring tab in the UI is showing weird things.  We triggered en_masse contact updates at 5:00pm yesterday and things chugged away for maybe an hour or two, some 200s and some not, but nothing super unsual except for the concurrent timeouts thing.  But the Monitoring tab stopped showing data at some point, maybe around 7:00pm.  But today, that data from later in the night IS in there.  So that's weird.  Wasn't there last night, but now it is.  Also, the calls to our webhook are just kind of sporadic.  A few batches were sent this morning but just kind of at odd times and not very many batches.

I'll stop there for now.  Can anyone help us out here?  Thanks!

[IsaacTakushi · Accepted solution]

Hi, @mainespotter.

Thanks for the detailed description!

Per the Webhook URL and rate limits section of this article, the "Event throttling" setting in your app's UI actually is what determines the max_concurrent_requestsvalue.

A limit of 6 per minute is quite low (I believe the default is 5 per second), and batch changes could easily exhaust this limit, triggering "Concurrent timeout" errors. If your endpoint can handle the traffic, I'd recommend significantly raising the "Event throttling" limit. 

The second issue you describe sounds strange, but I'm guessing this is a side effect of the fact HubSpot can't make the calls quickly enough due to the low concurrency limit. If you still see this behavior after raising the limit, certainly let me know.

[mainespotter]

Thanks for the reply.  It's funny that it didn't occur to me that the Event Throttling setting is what actually sets the max_concurrent_requests value, but your explanation makes perfect sense.

I will adjust that value and see what happens.  I have the same feeling as you that the second issue may be resolved when I make that adjustment.

Thanks again for the clear reply and I'll report back and mark it as the accepted solution if it does the trick.

[mainespotter]

That seemed to do the trick, thanks!