HubSpot Ideas

JaclynSchmitz

Ability to A/B Test with Sequences

It would be great to have the ability to easily a/b test with email sequences. Certain emails within the sequence may perform better with variable wording, but there is no easily way to test this. 

HubSpotからのアップデート
ステータスに更新: Delivered
December 01, 2023 08:12 AM

A/B Testing Sequences is now live! 

 

A/B testing is a great way to figure out what resonates best with your prospects and optimize your content for outcomes. Navigate to Automation > Sequences to create a new sequence or start testing an existing one. Within any email step, you'll see the option to add an A/B test. Sends will split automatically and you'll be able to monitor results for each version.

sequences-ab-testing.png

For more details on testing and best practices check out this knowledgebase article.  

 

Best,
Ana

August 29, 2023 05:35 AM

@KhalidK Great question. 

 

Given this example sequence: 

 

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

 

1. Email 1 will be split with 50 contacts receiving A and 50 receiving B. All 100 contacts would receive Email 2 (A). Email 3 will also be split with 50 contacts receiving A and 50 receiving B. 

2. Randomization is per step in the sequence. If a contact receives version A of Email 1, they still have a 50/50 chance of receiving A or B for Email 3. 

 

I hope this helps clarify! Please let me know if you have any additional questions. I'm also happy to connect on a feedback call if you'd like to discuss in more depth.  

ステータスに更新: In Planning
May 19, 2023 08:39 AM

Hi Community, 

 

Thank you so much for the feedback and perspectives on this post. We understand how critical this feature is to experiment and improve your sequence performance and I am excited to say the ability to A/B test emails in a sequence is currently in progress! 

 

I'll be in touch with more updates when we're further along and ready for beta testers. Please feel free to reach out with any questions or feedback. 

 

Best, 

Ana

125件のコメント
JoeBurchard
トップ投稿者 | Elite Partner

Awesome update from the HubSpot side - this feature is now available in public beta!

To gain access to this feature click on:

  1. Your icon in the top right of your HubSpot portal
  2. Click product updates
  3. Search for 'A/B Testing'
  4. Click 'Read More'
  5. Click 'Join Beta'

Screenshot 2023-08-16 at 10.25.59 AM.png

 

Alternatively, you can go to https://app.hubspot.com/whats-new/[INSERT YOUR PORTAL ID]/all?productUpdateId=13899225

 

Who has access:

Sales Enterprise, Sales Pro, Service Enterprise, and Service Pro accounts*

*Users with a legacy paid Sales Hub will not have access to this public beta.

KhalidK
メンバー

Would love some clarification on how the randomization works here:

 

If we have the following sequence and push 100 users through the sequence

 

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

 

1. Will all 100 users receive Email 2? Or only the users who saw variant A in Email 1?

2. Is the randomization fixed for the entire sequence or determined per step in the sequence?

    i.e Will all users who saw variant A in Email 1 also see variant A in Email 3? or can users who saw variant B in Email 1 get variant A in Email 3? This is really important to determine if we can only test copy or orders of emails.

 

 

adimarino
HubSpot製品開発チーム

@KhalidK Great question. 

 

Given this example sequence: 

 

Email 1 (A/B) -> Email 2 (A) -> Email 3 (A/B)

 

1. Email 1 will be split with 50 contacts receiving A and 50 receiving B. All 100 contacts would receive Email 2 (A). Email 3 will also be split with 50 contacts receiving A and 50 receiving B. 

2. Randomization is per step in the sequence. If a contact receives version A of Email 1, they still have a 50/50 chance of receiving A or B for Email 3. 

 

I hope this helps clarify! Please let me know if you have any additional questions. I'm also happy to connect on a feedback call if you'd like to discuss in more depth.  

KhalidK
メンバー

@adimarino 

 

This helps a lot - is there any way to randomize at the top of the funnel? We're trying to figure out how to A/B test sequences of emails i.e

 

Control (A,B,C,D)

Treatment(B,A,C,D)

 

Right now this means that in Email 2, some users could get Email A/B twice.

ステータスに更新: Delivered
adimarino
HubSpot製品開発チーム

A/B Testing Sequences is now live! 

 

A/B testing is a great way to figure out what resonates best with your prospects and optimize your content for outcomes. Navigate to Automation > Sequences to create a new sequence or start testing an existing one. Within any email step, you'll see the option to add an A/B test. Sends will split automatically and you'll be able to monitor results for each version.

sequences-ab-testing.png

For more details on testing and best practices check out this knowledgebase article.  

 

Best,
Ana