APIs & Integrations

ah8
Contributor

Workflow Custom Code Action Large Load Retries

SOLVE

Hi,
I have about 175,000 contacts about to go through a workflow and have a custom code action run on them. I've been testing it with 10,000, and I find that I keep seeing, "Custom code wasn't able to execute, but will retry soon" in the Information Event field. That should be fine, but after waiting quite a while, I notice that a lot of the fields that should have been filled in haven't. That is, some didn't make it through the workflow.

Any suggestions on how to do it?

Thank you in advance.

0 Upvotes
2 Accepted solutions
SteveHTM
Solution
Guide | Partner
Guide | Partner

Workflow Custom Code Action Large Load Retries

SOLVE

@ah8 - my sympathies on this. I have had several mysterious occurences of similar hang-ups when dealing with large volumes of custom code workflow invocations. My suspicion is that the interface to AWS Lambda behind the scenes can get bogged down. That's not much help in itself, but running through all your records is likely to require a lot of batch subdivision and speading over longer timescales. At least that's the only thing that's worked for me.

 

Steve

Steve Christian

HTM Solutions

https://info.htmsolutions.biz/meetings/stevec2

mobilePhone
+1 6195183009
emailAddress
stevec@htmsolutions.biz
website
www.htmsolutions.biz
address
San Diego, CA
Create Your Own Free Signature

View solution in original post

dsmarion
Solution
Top Contributor | Gold Partner
Top Contributor | Gold Partner

Workflow Custom Code Action Large Load Retries

SOLVE

If your custom code does not need to return anything to the workflow you could use a webhook action instead where you simply log the webhook in a database or queue system and then use scheduled tasks to process them a batch at a time.  If your custom code must execute and then return something to the workflow for further processing this becomes a bit of a nightmare imo.

Scott Marion
Senior Developer @ Thread Connected Marketing

View solution in original post

2 Replies 2
SteveHTM
Solution
Guide | Partner
Guide | Partner

Workflow Custom Code Action Large Load Retries

SOLVE

@ah8 - my sympathies on this. I have had several mysterious occurences of similar hang-ups when dealing with large volumes of custom code workflow invocations. My suspicion is that the interface to AWS Lambda behind the scenes can get bogged down. That's not much help in itself, but running through all your records is likely to require a lot of batch subdivision and speading over longer timescales. At least that's the only thing that's worked for me.

 

Steve

Steve Christian

HTM Solutions

https://info.htmsolutions.biz/meetings/stevec2

mobilePhone
+1 6195183009
emailAddress
stevec@htmsolutions.biz
website
www.htmsolutions.biz
address
San Diego, CA
Create Your Own Free Signature
dsmarion
Solution
Top Contributor | Gold Partner
Top Contributor | Gold Partner

Workflow Custom Code Action Large Load Retries

SOLVE

If your custom code does not need to return anything to the workflow you could use a webhook action instead where you simply log the webhook in a database or queue system and then use scheduled tasks to process them a batch at a time.  If your custom code must execute and then return something to the workflow for further processing this becomes a bit of a nightmare imo.

Scott Marion
Senior Developer @ Thread Connected Marketing