Ability to A/B Test with Sequences

[JaclynSchmitz]

It would be great to have the ability to easily a/b test with email sequences. Certain emails within the sequence may perform better with variable wording, but there is no easily way to test this. 

[JaclynSchmitz]

A/B Testing Sequences is now live! 

A/B testing is a great way to figure out what resonates best with your prospects and optimize your content for outcomes. Navigate to Automation > Sequences to create a new sequence or start testing an existing one. Within any email step, you'll see the option to add an A/B test. Sends will split automatically and you'll be able to monitor results for each version.

For more details on testing and best practices check out this knowledgebase article.  

Best,
Ana

[JaclynSchmitz]

@KhalidK Great question. 

Given this example sequence: 

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

1. Email 1 will be split with 50 contacts receiving A and 50 receiving B. All 100 contacts would receive Email 2 (A). Email 3 will also be split with 50 contacts receiving A and 50 receiving B. 

2. Randomization is per step in the sequence. If a contact receives version A of Email 1, they still have a 50/50 chance of receiving A or B for Email 3. 

I hope this helps clarify! Please let me know if you have any additional questions. I'm also happy to connect on a feedback call if you'd like to discuss in more depth.  

[JaclynSchmitz]

Hi Community, 

Thank you so much for the feedback and perspectives on this post. We understand how critical this feature is to experiment and improve your sequence performance and I am excited to say the ability to A/B test emails in a sequence is currently in progress! 

I'll be in touch with more updates when we're further along and ready for beta testers. Please feel free to reach out with any questions or feedback. 

Best, 

Ana

[JoeBurchard]

Awesome update from the HubSpot side - this feature is now available in public beta!

To gain access to this feature click on:

1. Your icon in the top right of your HubSpot portal
2. Click product updates
3. Search for 'A/B Testing'
4. Click 'Read More'
5. Click 'Join Beta'

Alternatively, you can go to https://app.hubspot.com/whats-new/[INSERT YOUR PORTAL ID]/all?productUpdateId=13899225

Who has access:

Sales Enterprise, Sales Pro, Service Enterprise, and Service Pro accounts*

*Users with a legacy paid Sales Hub will not have access to this public beta.

[KhalidK]

Would love some clarification on how the randomization works here:

If we have the following sequence and push 100 users through the sequence

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

1. Will all 100 users receive Email 2? Or only the users who saw variant A in Email 1?

2. Is the randomization fixed for the entire sequence or determined per step in the sequence?

    i.e Will all users who saw variant A in Email 1 also see variant A in Email 3? or can users who saw variant B in Email 1 get variant A in Email 3? This is really important to determine if we can only test copy or orders of emails.

[adimarino]

@KhalidK Great question. 

Given this example sequence: 

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

1. Email 1 will be split with 50 contacts receiving A and 50 receiving B. All 100 contacts would receive Email 2 (A). Email 3 will also be split with 50 contacts receiving A and 50 receiving B. 

2. Randomization is per step in the sequence. If a contact receives version A of Email 1, they still have a 50/50 chance of receiving A or B for Email 3. 

I hope this helps clarify! Please let me know if you have any additional questions. I'm also happy to connect on a feedback call if you'd like to discuss in more depth.  

[KhalidK]

@adimarino 

This helps a lot - is there any way to randomize at the top of the funnel? We're trying to figure out how to A/B test sequences of emails i.e

Control (A,B,C,D)

Treatment(B,A,C,D)

Right now this means that in Email 2, some users could get Email A/B twice.

[adimarino]

A/B Testing Sequences is now live! 

A/B testing is a great way to figure out what resonates best with your prospects and optimize your content for outcomes. Navigate to Automation > Sequences to create a new sequence or start testing an existing one. Within any email step, you'll see the option to add an A/B test. Sends will split automatically and you'll be able to monitor results for each version.

For more details on testing and best practices check out this knowledgebase article.  

Best,
Ana