Enable A/B testing in workflow/automated emails

I was disappointed to see that I can't do an A/B test in an automated email:

https://knowledge.hubspot.com/articles/kcs_article/email/how-to-create-an-a-b-test-email

 

Marketo (a HubSpot competitor) allows this functionality.

Can you add this to the product sometime?

It's particularly helpful with a Welcome email sequence which is key to on-boarding new customers.

HubSpot updates
Feb 14, 2020

Hi all! Two quick updates for you on this:

 

Number 1: Progress is going well, we have our new A/B framework in good shape and at the moment we're focusing on the new in-app features needed to begin/end and analyse this new type of A/B test for workflow emails. This is a big update on what is currently available for batch emails, and will give you the ability to end an A/B test and start a new one on the same email without needing to rebuild the email over and over again. Getting very close now! 

 

Number 2: Moving forward this feature, the updates on it's release and all feedback will be handled by my colleague @jaleonard19. Jake is also a PM here working on our marketing email tools and has actually been the one spearheadding this feature for some time now so you'll be in great hands. 

 

Jake is currently working on our release plans and will be sharing a public beta signup form with you all as soon as it's ready. Thank you all again for your feedback and suggestions. Looking forward to seeing the impact this will have for you all.

Dec 19, 2019

Will do @petersherlock93

Dec 11, 2019

Will do @EJVFernyhough ! We'll be sharing a beta sign up form with everyone on this post when it's available so everyone will have the opportunity to sign up once it's open.

Dec 11, 2019

@AJernigan can do Smiley Happy

Nov 18, 2019

Hi @ridingforlife I can definitely share more. At the moment we're making good progress but factoring in the upcoming US and EU holiday season it's unlikely that we'll be hitting beta-ready in Q4 but safe to say that we'll have a beta in Q1 2020. As soon as we have an open beta we'll offer it up to everyone in this thread to give it a try. 

Nov 4, 2019

Hi all, just to keep everyone updated on our progress here, we've been making decent headway with the infrastructure planning and development roadmap for this feature and are making meaningful progress towards a beta.

 

There's still a lot of moving pieces here so we don't have a concrete date on when we'll be opening up a beta but I wanted to keep you all in the loop and confirm that we're definitely focused on this and are actively working on it.

Aug 14, 2019

Thanks @petersherlock93! Just a case of good ol' miscommunication Smiley Happy  The "on pause" here just means that we were still deep into the infrastructure and reliability work needed for this (and other) functionality. It was by no means deprioritised or taken off the roadmap, just needed to get the ordering right on the priorities to make sure we didn't put the cart in front of the horse, so to speak.

Aug 13, 2019

Hi @petersherlock93  (and all following along). I can confirm that this isn't the case, A/B testing of automated emails is still very much a priority for us.

 

For the past few months we have been focused primarily on work on the underlying infrastructure of email, to ensure speed, stability and reliability of the overall platform and user experience. We're nearing the end of this work right now which enables us to continue to build on top of the email tool and makes it possible for us to build more net-new functionality in a stable and scalable way, A/B testing of automated emails being a great example of that. 

 

While there hasn't been much news on this specific feature lately, it's something that is very high on our roadmap and now that we're seeing results from our infrastructure and performance work we're able to refocus our efforts on bringing these new features to the product. Rest assured that it's still something we're focused on. 

 

While we aren't yet at the stage of a beta for this feature, I'd recommend checking out these resources (one and two) from the HubSpot blog that provide interim solutions and as soon as we have a beta that's available for everyone I'll share a signup form for everyone here to try out. Thanks again for your feedback and for keeping the momentum going here. 

changed to: In Planning
Jan 29, 2019

Hi all, I wanted to share a quick status update with you on this. We've spent some time researching options for this feature and are making good progress. We'll soon be heading into a dedicated research phase and as we get close to a beta of this we'll open it up to everyone on this post to get early access. Please do keep your feedbac/upvotes coming, we really appreciate it.

Nov 16, 2018

Thank you all for sharing your feedback and use cases on this. The product team is currently thinking about how to solve this, and will post updates here. 

Apr 6, 2017

A/B testing an entire workflow to see what places could be improved to improve conversion rates.

220 Replies
New Contributor

Registered for the forum just to comment on this; this is deal-breaking functionality as far as I'm concerned.  It's available in Marketo and Salesloft, as well as many other email marketing platforms.  It's required, and to see that it's requested for years and not implemented is extremely disappointing.

 

I'll likely be looking at alternative platforms to solve this.

New Contributor

@hubspot

 

Even Mailchimp which is an entry level ESP has this feature.

New Contributor

Looking at HubSpot as an alternative to Intercom. This feature is kind of a deal breaker.

Occasional Contributor

Has there been a solution created yet for this? 

 

It's very manual work if we have to turn on and off emails every time we want to test a new feature, subject line, content etc in a workflow. 
And if I do an A/B test on certain elements in a single-send email, it's not reliable data for me to fall back on since my audiences would have different intents. Help, Hubspot!

Occasional Contributor

+1 Agreed! Please add this feature. It's a must-have. 

Occasional Contributor

I do understand the rationale that automated emails send at different times thus you couldn't definitively say that version A is truly the winner, b/c we can't control time. However, if an automated email has been sent to 300 contacts, we can assume that across the 2 different a/b pools the submission times have averaged out - and thus attribute the success to the corresponding version.

That being said - the proposed option of sending out manual emails until we determine a winner is the least helpful "solution" ever. I'm sure MANY hubspot users have automated workflows that are triggered upon a new contact/lead submitting their information. To have someone on hand, ready to send an email as soon as someone has visited their website and provided their information is ludicrous. FURTHERMORE, they would be sending 1-off emails, and no simple way to actually track and analyze the success of the emails.

 

I still continue to be shocked by the seemingly obvious implementations and functions that Hubspot does not offer.

Currently the no automated email a/b testing option is worse than the option to a/b test automated emails, knowing that the results aren't necessarily 100% accurate.

Hubspot should make this one of their top priorities.

 

New Contributor

I found this on the blog and tried out the first option. https://blog.hubspot.com/customers/how-to-a/b-test-workflow-emails-using-the-lead-rotator-or-zapier

 

Takes a little bit of time to set up, but seems to be working properly so far

Community Manager
Community Manager

<Posting this on behalf of a customer>

 

A/B test in workflow will be a great feature because we would like to do A/B test including the actions in the workflow.

If this will be added as a feature in the portal, we would be able to test not only the contents of the emails but also the actions in the workflow.

New Contributor

really want that. i'd like to know which set of step emails is better.

Occasional Contributor | Platinum Partner
Occasional Contributor | Platinum Partner

This is an absolute must-have!

Occasional Contributor

There are a few things HubSpot is missing from a basic functionality standpoint...and this is one of them...Please add this functionality. 

Occasional Contributor

We need this badly, any update on when it will be available?

Occasional Contributor

Please hubspot please!  You're making us look bad... Smiley Wink

Visitor

I don't understand how Hubspot being a platform so complete, it doesn't have this simple feature. EVERY sales automation platform I used/tested have this feature.

It is something that can make me decide to change my sales automation provider.

New Contributor

 I thoroughly support this idea.  Seems like an obvious enhancement, and one that would get a lot of use.

Regular Contributor

This is a BASIC feature!! Plase add it asap

New Contributor

Very surprised to find that this is not currently possible. Especially for simply testing subject line variations.

Occasional Contributor

How is this still not a feature? So many have been requesting it for the last year and a half! Please prioritize.

New Contributor

In the meantime while they absolutely need to sort this out. You could use properties to set up variables. I used the ownership field and set a workflow to auto-assign to one of 2 owners in a rotation. This gives you a 50-50 split test group. This also requires you to not be using the CRM as a CRM, however.

 

I imagine you can use a bit of excel magic to assign values similarly and append them to your email file to create the A/B lists.

HubSpot Product Team
HubSpot Product Team

Thank you all for sharing your feedback and use cases on this. The product team is currently thinking about how to solve this, and will post updates here.