Enable A/B testing in workflow/automated emails

I was disappointed to see that I can't do an A/B test in an automated email:

https://knowledge.hubspot.com/articles/kcs_article/email/how-to-create-an-a-b-test-email

 

Marketo (a HubSpot competitor) allows this functionality.

Can you add this to the product sometime?

It's particularly helpful with a Welcome email sequence which is key to on-boarding new customers.

258 Replies
New Contributor

@bobaloobadoo you can create additional user property (i called mine `test_group`) and set that field in your workflow rotation to keep `Contact Owner` usable in CRM.

HubSpot Employee

This would be a really great feature! +1 This Idea

Occasional Contributor

Totally +1 this idea. 

Your logic behind not being able to AB test on automated emails is completely flawed - following the same logic, Google should not also allow this kind of tests on their ads!

It's a complete fail on your part, Hubspot, that this functionality is unavailable.

Regular Contributor

Have to agree this reply makes no sense. 

 

You enable a/b testing on landing pages, but people visit landing pages at different times of day. 

 

If it makes sense to A/B test ad content or landing pages which deliver at different times of day to individual people, it 100% makes sense to A/B test emails in a flow.

 

It's about improving the flow, and if you are sending a large volume of emails the time of day issue should not skew the data. 

New Contributor

This would be a very beneficial addition to HubSpot to be able to A/B Test in a workflow. We have several onboarding email workflows for new customers and being able to test different emails for new users would be helpful.

Occasional Contributor

Adding my voice fo support to this thread in the hopes this becomes a feature in the near future. The inability to A/B split an automated email seems like a massive feature gap to me. the value of Hubspot to me woud be dramatically increased if A/B slitting automated emails was a feature. 

New Contributor | Platinum Partner

+ 1 we really need this feature for us and our clients 

 

I follow this article : https://blog.hubspot.com/customers/how-to-a/b-test-workflow-emails-using-the-lead-rotator-or-zapier but this is complicated to set up 

Occasional Contributor

Agreed - this makes no sense and not having A/B test means there is no way at all of testing a change (i.e. a text link or a button link). There may be a point regarding different times of day, but without any form of testing there is nothing at all to test against an we are in the dark of any changes made.

 

Going to try out the following: https://blog.hubspot.com/customers/how-to-a/b-test-workflow-emails-using-the-lead-rotator-or-zapier as some form of A/B testing is paramount to us evolving email 

Occasional Contributor

this is non-sensical. 

this needs to be implemented.

Status updated to: In Planning
HubSpot Product Team

Hi all, I wanted to share a quick status update with you on this. We've spent some time researching options for this feature and are making good progress. We'll soon be heading into a dedicated research phase and as we get close to a beta of this we'll open it up to everyone on this post to get early access. Please do keep your feedbac/upvotes coming, we really appreciate it.

Occasional Contributor

Thank you Shay, that's lovely to hear!

Occasional Contributor

This is the single largest feature gap of HubSpot. We could literally learn so much, so much faster if this was a feature. The day this gets released I will sing delighted praises to the name of Dharmesh Shah. 

 

Jorgen

Occasional Contributor

I think this feature is really missing. I would be absolutely necessary for us.

Occasional Contributor

This is a MUST for us. Please prioritize it 

New Contributor

+1 for this Idea... its totally necessary!!

New Member

+1 - Seems so basic.  I saw another tool that handled it beautifully.  Hubspot is a ltech eader and I expected to find it  in the tool.

New Contributor | Platinum Partner

I hate that HubSpot doesn't offer this feature as well. 

 

I am constantly working to optimize my workflows and the best way I have found to do it is super manual but it has been working. 

 

I write down the day I implemented an email change and monitor it on a monthly basis until it reaches statistical significance using this calculator: https://www.verticalmeasures.com/resources/ppc/statistical-significance-calculator/ 

 

Once the test has reached at least 95% statistical significance I conclude if the test won or was unsuccessful before determining if I will keep this change in place. 

 

It's not the most efficient but it has been working for me! 

Occasional Contributor

This is great to see in the planning and research phases, it's a huge need for our email marketing campaigns. 

Occasional Contributor

When will Hubspot be implementing this feature? If outreach.io, marketo, etc. can do it, then surely Hubspot can!

-Angela

New Contributor

I've just set a workflow up using an IF/THEN split and duplicated automated emails with different subject lines (or whatever other element you wish to test).

 

IF "email" starts with "a" through to "m", they get Version 1. If it's "n" through to "z", they get Version 2.

 

It's probably not the best A/B testing methodology, but it should work.