What’s an A/B test you’ve performed that surprised you and why?

[chrlee]

Tell us about an A/B test you've conducted in email marketing. Give us some context about your company and industry, then tell us about the parameters of your experiment. What were the results and why did they surprise you?

[hparvi]

An A/B test comparing "Buy Now" vs. "Add to Cart" CTAs revealed that "Add to Cart" increased click-through rates by 25% and purchases by 10%. This surprising result showed that users preferred the lower-commitment option, highlighting the importance of testing small changes and understanding user psychology.

[DGay]

I have yet to do any a/b testing, but as I am now able to focus on my email marketing because I have my day-to-day operations covered in my small business, I plan on a/b testing a lot. I want to begin with click-through rates, as my open rate is usually well above the average. 

[RSumner]

New here, still learning and processing all the information which is a wonderful resource for when I do utilize this!

[Haleemah_01]

I'm yet to perform an A/B test in Email Marketing. I am looking forward to doing that after completing this course

[AreejHsn]

I'm still learning about A/B testing😊

[BeccaVL]

I am yet to conduct A/B Testing in my current business, as we are not yet in the position to start sending email campaigns. However, previously I have tested different types of subject lines and call to actions - the results have always proven the more direct, clear and to the point, the better.

[BBlaise]

I am new here currently taking and understanding the concepts

[PAdunmoye]

I'm still learning the ropes

[Graciouslygrace]

I'm still learning about A/B testing 

[Pamilerin_03]

I am still learning about the A/B testing. 

[Zufar]

I ran an A/B test for an email campaign: Variant A had a formal subject line, 'We Miss You! Here’s an Exclusive Offer,' while Variant B used a casual tone, 'Hey, [Name], Let’s Catch Up!'

Surprisingly, the casual tone boosted open rates by 35% and click-throughs by 20%, showing that a personal approach resonated better with my audience.

[nvasquez13]

What I learn from A/B testing is that, a lot of times, as a marketer what I thought was a good and beautiful copy or content are not going to work out there for conversion, open rates etc. These are the reason why we should A/B testing to determine if the copies, strategies, etc are going to work. This is saving me cost and time before I send it to the whole e-mail list.

[Kayohde]

I haven't done this yet, still learning, I'll soon try it

Probably next week

[JGood1]

I haven't had the ability to do this yet as I'm still in school, but hopefully I will be able to do this for a job that I'm applying for. 

[JKotzman]

I am learning about the A/ B testing in this course and I am a college student, so hopefully the knowledge I will gain will help me in my future career pursuits.

[PThompson70]

We added an emoji to the subject line and saw nearly a 10% point increase in open rate vs. emails w/out the emoji! Emoji are cute so the surprise was how it seemed to resonate with the audience. 

[TMagaisa]

Why It Worked:

1. Attention-Grabbing: Emojis stand out in a crowded inbox
2. Emotional Connection: Emojis can convey emotions and create a more personal connection, which can be particularly effective in building rapport with our audience.
3. Curiosity: The presence of an emoji might have sparked curiosity, prompting recipients to open the email to see what it was about.

Takeaways:

• Test and Learn: This experience reinforced the importance of continuous testing and being open to unexpected results. What might seem like a small change can have a significant impact.
• Audience Understanding: It also highlighted the need to understand an audience's preferences and what resonates with them. In this case, a simple emoji made a big difference.

I’d love to hear about any surprising A/B tests you’ve conducted and the results you’ve seen. What unexpected insights have you gained from your experiments?

[destinyatdefy]

I'm new to email marketing, so I'll use an example from content creation instead. I am a solopreneur that helps micro sized companies effectively communicate their message on LinkedIn. Within 2 weeks, my clients report landing thier largest client to date, all thanks to a simple change in messaging!

One of my clients is building a really cool sales tool. But as an engineer, he struggles to clearly communicate his business' value to prospects. He also struggles to position himself as an authority figure in sales/AI and because of this, he didn't feel comfortable making "salesy posts."

I listened to his comments and understood we had to try something different. His views on LinkedIn were plummeting. 

We created a test:

A: Adding a clear call to action in the last sentence of his post

B: Softly inviting people to click the link in his bio to learn more

Both were posted on a Wednesday, 9 AM CST. I was surprised by how well both posts performed, but A outperformed B almost 2:1. Within a week, the CTR (click through rate) on his website was significantly higher than the following week. No other variables changed, so we concluded that it was best to add a clear call to action at the end of his posts as opposed to "kindly inviting" his audience. 

[OOlaitan]

One good example is testing subject lines. Most marketers would think that personalized subject lines-say, populating a contact's first name-would beat out more straightforward or creatively curious subject lines. But then the simple, non-personalized subject lines end up performing better for open rates. That's surprising, considering personalization is usually such a strong engagement tactic, but sometimes it can come off as overly automated or forced. Takeaway: A/B testing helps to unravel what works best for your unique audience.

[prabhukiran]

Hi,

I am still in learning phase, will update here if I notice any difference in CTR or Conversions etc, when I start an email campaign with A/B testing. 

[lKaur5]

One example is testing subject lines. Many marketers expect that using personalized subject lines (e.g., including a contact’s first name) will outperform more straightforward or curiosity-driven ones. However, sometimes the simpler, non-personalized subject lines perform better in terms of open rates. This can be surprising because personalization is typically seen as a strong engagement tactic, but it can backfire if it feels too automated or forced. The takeaway is that A/B testing helps uncover what works best for your unique audience.