A/B Testing Questions

Marketing & Content Hub Email Marketing Tool
1

Hi, I’ve got a couple of questions regarding the A/B testing functionality within HubSpot.

  1. Why are we limited to only A/B tests?
  2. How are we expected to set the distribution % if we can’t see the list size?
  3. Why is there no option to manually select the test winner?
  4. Why when selecting ‘open rate’ as the testing criteria, which it specifies as ‘subject line, from name, preview text and from address’, does it generate a second version of the email, that then has to be updated and mainted?
2

Hi @Joe_J,

Happy to help here.

@Joe_J wrote:
Why are we limited to only A/B tests?

HubSpot apparently hasn’t prioritized this so far, if you’re referring to multivariate testing. Other users have requested this before: https://community.hubspot.com/t5/HubSpot-Ideas/Allow-a-multivariate-of-A-B-C-D-tests-on-marketing-emails/idi-p/1003500

These requests submitted to the HubSpot Ideas section of the community are reviewed by the HubSpot product team, based on their popularity and the assumed demand. I’d recommend commenting and upvoting.

You can also help other HubSpot users find this request more easily (and drive traction) by accepting my reply as a solution. I’d appreciate it, too.

@Joe_J wrote:
How are we expected to set the distribution % if we can’t see the list size?

You can always check the list size by leaving the A/B test configuration and going into the send or schedule tab. The UI could be optimized, yes. If you feel strongly about that, you could submit a request in the HubSpot Ideas section of the community.

@Joe_J wrote:
Why is there no option to manually select the test winner?

This, too, has been requested before and apparently hasn’t been a priority for the product team yet: https://community.hubspot.com/t5/HubSpot-Ideas/End-A-B-Test-on-Marketing-Emails-Early/idi-p/369163

@Joe_J wrote:
Why when selecting ‘open rate’ as the testing criteria, which it specifies as ‘subject line, from name, preview text and from address’, does it generate a second version of the email, that then has to be updated and mainted?

HubSpot made the design choice to treats these as separate assets. If you’re concerned about changes that have to be made in both variants, I’d recommend always creating the A/B test in the very last minute.

Clearly you’re asking these questions because you’re not happy with how the A/B test tool is currently designed. The best place to provide this feedback, see above, is through the HubSpot Ideas section of the community or through your customer success manager.

Have a great day!

3

Solid response from @karstenkoehler as per usual