Single-Send Email A/B Test Workaround for <1,000 Recipients

SOLVE
Highlighted
Occasional Contributor

Hello Hubspot Community!

 

I am wondering what the easiest workaround would be to send a single-send A/B test for under 1,000 recipients using an active list.

 

I understand that <1,000 people is ~not~ statistically significant. However/for better or worse, it's what I have to work with given the narrow segmentation of our emails, and am looking to get some insights.

 

I imagine it's something like making the active list static, exporting, splitting, etc. but would love to hear from someone who has done this and has a relatively easy/efficient solution.

 

Thank you,

Jamie

Reply
0 Upvotes
1 Accepted solution

Accepted Solutions
Advisor

There's nothing special you have to do.  While more than 1k recipients is needed for statistical significance, it is not needed to run an A/B test.  Unfortunately what would happen is that the "winning" version would always be your fallback version. Or you can do away with a winning version altogether and just do an A/B with 100% of the recipeient and analyze your results after to continue to improve your email efforts.

 

hubspot_abtest.jpg

2 Replies 2
Advisor

There's nothing special you have to do.  While more than 1k recipients is needed for statistical significance, it is not needed to run an A/B test.  Unfortunately what would happen is that the "winning" version would always be your fallback version. Or you can do away with a winning version altogether and just do an A/B with 100% of the recipeient and analyze your results after to continue to improve your email efforts.

 

hubspot_abtest.jpg

Occasional Contributor

Thank you! Yes, that makes sense - I was doing an A/B partial split, hence why it only sent the "winning" result.

Reply
0 Upvotes