HubSpot Ideas

katharine

Enable A/B testing in workflow/automated emails

I was disappointed to see that I can't do an A/B test in an automated email:

https://knowledge.hubspot.com/articles/kcs_article/email/how-to-create-an-a-b-test-email

 

Marketo (a HubSpot competitor) allows this functionality.

Can you add this to the product sometime?

It's particularly helpful with a Welcome email sequence which is key to on-boarding new customers.

Mises à jour HubSpot
Statut mis à jour : Delivered
November 18, 2021 08:35 AM

Hi All,

 

We recently rolled this out LIVE to all pro+ portals. Thank you for all your patience and interest in this project.

 

Best,

 

Shane Janssens

Statut mis à jour : In Beta
January 13, 2021 04:51 AM

Hi Everyone,

 

Thank you for your pateince here. I am ecstatic to be updating this post to "In Beta" today. We rolled this feature out to a list of Beta users from this morning. If you want to be added to it please reach out to support and they can request it through the proper channels. 

We know this has been a very popular request and the team have been working so hard to deliver this as fast as possible. We hope to be rolling it out to a wider audience in coming weeks.

Best,

 

Shane

Statut mis à jour : Being Reviewed
December 02, 2020 05:25 AM

Statut mis à jour : Idea Submitted
November 23, 2020 03:50 AM

Hi All! Again thank you for your patience on this feature. Again we are almost there with some final checks and tests. We hope to have this released to Beta next week. I will update this post when it is live. Please do reach out to your CSM if you want to be part of the Beta - By this I mean send an email to support and they can add you to the Beta list.

 

We are really excited to get this out after months of hard work by the team here.

 

Thank you!


Best,

 

Shane Janssens

November 13, 2020 08:56 AM

Hi All, 

I just wanted to update the recent questions @petersherlock93 , @jmcguire20 , @stuartborders , @JPD123 

 

We are in final testing here. expecting to release to beta group in late Nov. If you want to get siogned up for Beta please reach out to your CSM and they can add you to list.

 

I commented with updates but they seem to be lost in thread. Sorry about confusion.

 

Best,

 

Shane

September 30, 2020 01:36 AM

Hi all,

 

Quick update here. Again thank you for your patience and feedback here. We are currently moving to testing this feature. It will take a bit of time as it is a big change on the email infastructure. I will update here as soon as it is in Beta. I wont give an exact timing as it depends on what we find in testing. Anyone who has signed up will be in the first release group, if you have not asked support for this please reach out and they can add you to the beta list.

 

Thank you, it is very close now.

 

Best,

 

Shane

 

Statut mis à jour : Being Reviewed
September 09, 2020 06:07 AM

Hi Everyone,

 

Thank you so much for all your interest and comments in this forum. I am the Product Manager on Marketing Email and I wanted to jump in here as I have been closest to this project for past few months. We are so close, we had hit a couple of small bumps that set us back with the complexity of this build. We are in the final stretch and we are talking about delivering this in September. We have had 2 teams involved in building this and we are so excited to release the Beta as soon as we can. I will update this post as soon as it is ready for beta.

Thank you for your patience and continued interest in this project.

 

Best,

 

Shane

July 24, 2020 05:46 AM

Hi folks - Jumping in here again from the HubSpot Product team. I want to apologize for how long this has taken. This work is in progress and is a high priority for the team. Unfortunately, we have hit a number of snags during the development of this, which is slowing down the release. Hopefully in another month or so we should have clarity on timing. I'll post here as soon as we have more details to share. In the meantime please feel free to keep commenting, and upvoting the post.

April 20, 2020 01:58 PM

@btuttle279 Absolutely - just sent over a message. We can jump on a call and walk through what this will all look like! 

April 20, 2020 12:57 PM

Hi folks - Just posting a quick update on this thread. We are still focused on this project. Waiting on a couple of the backend changes to wrap up before we move into a Beta. Unfortunately, the work took a bit longer than originally anticipated. We are targeting getting the feature into a Beta this quarter (hopefully sometime in late May/ early June). As we roll out the Beta I will announce that in this forum, and post a form for you all to fill out to join. So hang tight for that!


We are very excited to get this one rolled out to you all and to start collecting feedback on it.

February 26, 2020 09:16 AM

Hi Folks - 

As @Shay mentioned I will be taking over this thread. I wanted to post an update and ask a couple of questions to the group here. Here is the basic functionality we will be supporting in the Beta:

  • A simple 50-50 split between 2 variants 
  • Ability to start and end tests on the same automated email (without cloning and republishing). 
  • Ability to see a history of the tests run on that automated email. 
  • Manually choosing a winner based on open/ click data. 

When digging into this project we realized that AB testing for Automated emails differs from regular 1-off batch emails. The main difference is that Automated emails have the potential to last forever. Thus we needed to support the ability to start and end multiple tests on an Automated email. Tracking those changes and tests over time. Here are some questions I had for the group:

 

Q1: How do you all keep track of the different experiments and tests you have run today with Batch Email? Or with other tools (website pages, CTAs, etc.)?  

Q2: Are their other metrics that are important to picking a winning variant other than Open & Click information? What are they and how do you solve for that today?

Q3: What are the other important pieces of functionality that are missing from above? And which are the most important to you? (ex/ configurable percentages, multivariant, calculations to pick a winner, ability to have HS pick a winner, etc.) 

 

We are getting very close to having a Beta. Once we have it I will post a form in this thread in order for you all to join. We will then be quickly iterating and adding more advanced functionality.  

 

February 14, 2020 08:09 AM

Hi all! Two quick updates for you on this:

 

Number 1: Progress is going well, we have our new A/B framework in good shape and at the moment we're focusing on the new in-app features needed to begin/end and analyse this new type of A/B test for workflow emails. This is a big update on what is currently available for batch emails, and will give you the ability to end an A/B test and start a new one on the same email without needing to rebuild the email over and over again. Getting very close now! 

 

Number 2: Moving forward this feature, the updates on it's release and all feedback will be handled by my colleague @jaleonard19. Jake is also a PM here working on our marketing email tools and has actually been the one spearheadding this feature for some time now so you'll be in great hands. 

 

Jake is currently working on our release plans and will be sharing a public beta signup form with you all as soon as it's ready. Thank you all again for your feedback and suggestions. Looking forward to seeing the impact this will have for you all.

December 19, 2019 01:57 AM

Will do @petersherlock93

December 11, 2019 03:32 AM

Will do @EJVFernyhough ! We'll be sharing a beta sign up form with everyone on this post when it's available so everyone will have the opportunity to sign up once it's open.

December 11, 2019 02:40 AM

@AJernigan can do Smiley heureux

November 18, 2019 07:12 AM

Hi @ridingforlife I can definitely share more. At the moment we're making good progress but factoring in the upcoming US and EU holiday season it's unlikely that we'll be hitting beta-ready in Q4 but safe to say that we'll have a beta in Q1 2020. As soon as we have an open beta we'll offer it up to everyone in this thread to give it a try. 

November 04, 2019 09:20 AM

Hi all, just to keep everyone updated on our progress here, we've been making decent headway with the infrastructure planning and development roadmap for this feature and are making meaningful progress towards a beta.

 

There's still a lot of moving pieces here so we don't have a concrete date on when we'll be opening up a beta but I wanted to keep you all in the loop and confirm that we're definitely focused on this and are actively working on it.

August 14, 2019 02:27 AM

Thanks @petersherlock93! Just a case of good ol' miscommunication 🙂  The "on pause" here just means that we were still deep into the infrastructure and reliability work needed for this (and other) functionality. It was by no means deprioritised or taken off the roadmap, just needed to get the ordering right on the priorities to make sure we didn't put the cart in front of the horse, so to speak.

August 13, 2019 10:39 AM

Hi @petersherlock93  (and all following along). I can confirm that this isn't the case, A/B testing of automated emails is still very much a priority for us.

 

For the past few months we have been focused primarily on work on the underlying infrastructure of email, to ensure speed, stability and reliability of the overall platform and user experience. We're nearing the end of this work right now which enables us to continue to build on top of the email tool and makes it possible for us to build more net-new functionality in a stable and scalable way, A/B testing of automated emails being a great example of that. 

 

While there hasn't been much news on this specific feature lately, it's something that is very high on our roadmap and now that we're seeing results from our infrastructure and performance work we're able to refocus our efforts on bringing these new features to the product. Rest assured that it's still something we're focused on. 

 

While we aren't yet at the stage of a beta for this feature, I'd recommend checking out these resources (one and two) from the HubSpot blog that provide interim solutions and as soon as we have a beta that's available for everyone I'll share a signup form for everyone here to try out. Thanks again for your feedback and for keeping the momentum going here. 

Statut mis à jour : In Planning
January 29, 2019 01:39 PM

Hi all, I wanted to share a quick status update with you on this. We've spent some time researching options for this feature and are making good progress. We'll soon be heading into a dedicated research phase and as we get close to a beta of this we'll open it up to everyone on this post to get early access. Please do keep your feedbac/upvotes coming, we really appreciate it.

November 16, 2018 02:07 PM

Thank you all for sharing your feedback and use cases on this. The product team is currently thinking about how to solve this, and will post updates here. 

April 06, 2017 05:05 PM

A/B testing an entire workflow to see what places could be improved to improve conversion rates.

293 Commentaires
Sandy
Membre

I've built out a free trial workflow for new users who sign up for our community and would like to to do some A/B testing of the emails ( starting with a simple subject line test). 

 

I've noticed that once I have activated my workflow I am not able to add an A/B test in the email builder. Is there a way to add this in without pausing the workflow? 

 

Thanks,

 

Sandy 

cbsembler
HubSpot Employee

Hello @Sandy

 

Currently, you won't be able to create an A/B test with an email that is saved for automated workflows. Make sure your email is set to send at a specific date/time or immediately (under the Sending tab in your email). 

 

The reason behind this is an automated email will continue to send over time at different intervals depending on when someone gets enrolled in the workflow. I would suggest if you wish to test some variables of your email you run an A/B Test on a regular single-send email and then depending on the results could implement that functionality into your automated email. 

 

Best, 

Courtney Sembler 

Inbound Professor 

bellawu
Équipe de développement de HubSpot

A/B testing an entire workflow to see what places could be improved to improve conversion rates.

arthurmaitre
Membre

Agreed - this is some must-have functionality if you ever want to optimize your automated email workflows.

 

Even just A/B testing the subject line would be helpful to maximize open rates.

roo12
Membre

We would like to A/B test the effect that sending out special offer emails to our database has on open rates and conversions. The best way we can think of is if there was a way to randomly put, for example, 1 in 10 subscribers into a separate static list that can be excluded from special offer emails. 

 

Another way this could be done is if we were able to pull subscribers from a number attribute (for example year of birth) based on 'number ending with X' which should give a relatively random selection. 

lilliclark
Membre

+1.  This seems absurd that it is not possible.  Many of our 'most important' emails are saved for automation in workflows.  It's non sensical that we can't optimize them.

Kateb
Participant

++++1 I can't believe Hubspot has not prioritsed this as a feature! An A/B testing functionality is an aboslte must for our automated emails! 

twandeleijer
Participant

I was really surprised to find out that this is not a feature in Hubspot. I thought Hubspot was all about automating the lead nurturing process. Without the ability to optimize the emails we use in our automated lead nurture workflows, there is really no way to optimize this workflow. Very dissatisfying..

JesRichards
Participant

I completly agree with this. Being able to test out different parts of a workflow could mean results are drastically changed. This would be a great feature to have. 

mcostet
Membre

I have the same issue. I think we constantly need to improve the messages we delivered our customers so A/B testing in automated emails is a must.

Hubspot you should develop that feature

AdrianB
Participant

Same boat. I would really appreciate this feature becoming a reality as it would multiply returns on time and fluidity tenfold.

StevenB
Participant

We really need this feature

leslie
Membre

I would love to have this feature too! Please prioritize 🙂

 

Thanks!

davidoudiette
Participant

It seems to make very little sense for A/B testing to be availabe anywhere but in workflows, where they get their whole potential. I've been advised to test complete workflows against each other, but setting it up is pretty much a nightmare compared to creating A/B test variations in an email. 

twandeleijer
Participant

@davidoudiette Imagine if you have a workflow that contains 2 emails, that would means testing 4 workflows. Or 3, that would mean 8 workflows. You can do that on the rest Smiley heureux

ckindrick
Membre

ditto. 

Emina-Demiri
Participant

Complete nightmare this. Every time I want to test a single email in a workflow I need to replicate the workflow and test it that way. It's not even easy to do that. You need to create new properties and add custom HTML to form embeds. Its shocking that this is not a simple functionality. 

 

Please add this

janik
Participant

Came here to find out how to set up A/B testing for my worklfows. Really disappointing (again) to see this obvisous feature is not implemented yet.

mecalagill
Contributeur

I totally agree because we are always updating our workflows to better serve our leads and clients. Even for just the beginning of an email workflow if we had the A/B testing options then we could see which one was performing better and that we could keep for our workflow. We are NOT going to get that info from a one-off email if it is in our workflow. That just doesn't make sense. 

sravi
Participant

Agreed! I had  submitted the same question to Hubspot. Not being able to see which emails are are doing better, what's working whats not is important. These workflows are a big part of my email automation. Checking single emails doesn't give me any insight at all. I specifically create emails that are a part of that workflow only. Also, having ability to see which clients are using a majority of the platform would make my life easier. Right now, there are so many testing browsers. Designing an email in a certain browser may not be optimal for viewing for the other. This is time-consuming and I have to make allowances for a lesser design. So, having that feature which tells me whats the majority of the population using to view these emails - will be a great thing!