Email Marketing

chrlee
HubSpot Employee
HubSpot Employee

What’s an A/B test you’ve performed that surprised you and why?

Tell us about an A/B test you've conducted in email marketing. Give us some context about your company and industry, then tell us about the parameters of your experiment. What were the results and why did they surprise you?

376 Replies 376
Dagevi93
Member

What’s an A/B test you’ve performed that surprised you and why?

I am new to A/B testing and am currently learning about it.

0 Upvotes
EOmusheni
Member

What’s an A/B test you’ve performed that surprised you and why?

I am new to Email marketing, so I have not conducted an A/B test yet.

 

0 Upvotes
KBoateng
Participant

What’s an A/B test you’ve performed that surprised you and why?

I am new to email marketing , so I have not conducted an A/B test yet

0 Upvotes
Debopam
Member

What’s an A/B test you’ve performed that surprised you and why?

Same here

 

dhananjayingole
Member

What’s an A/B test you’ve performed that surprised you and why?

Sure, let's say you work in the fashion retail industry and your company sells clothing online. You could conduct an A/B test on your email marketing campaign by testing two different subject lines:

Version A: "Exclusive Spring Collection Now Available!"
Version B: "Shop Our New Spring Arrivals Today!"

After sending these emails to two equal segments of your subscriber list, you might find that Version B ("Shop Our New Spring Arrivals Today!") resulted in a higher open rate and click-through rate compared to Version A. This suggests that the more action-oriented subject line performed better in engaging subscribers. As a result, you might decide to use similar language in future email campaigns to optimize engagement and conversions.

PArotiba
Member

What’s an A/B test you’ve performed that surprised you and why?

Engaging your email analytics can help you understand your audience better, improve contacts’ engagement, increase your revenue, and optimize to scale.

0 Upvotes
Tianakris
Member

What’s an A/B test you’ve performed that surprised you and why?

I am new to A/B testing and am currently learning about it.

0 Upvotes
KellyLe02
Member

What’s an A/B test you’ve performed that surprised you and why?

I have not done it yet but I thought about a scenario where a clothing retailer, "FashionFusion," is conducting an email marketing campaign to promote their new summer collection. The company specializes in trendy, affordable clothing for young adults and operates primarily in the e-commerce sector.

A/B Test Parameters: FashionFusion decides to conduct an A/B test to determine the effectiveness of different subject lines in their promotional email. They send out two versions of the email to their subscriber list:

Version A: Subject Line - "Summer Style Essentials: Shop Now!" Version B: Subject Line - "Hot Deals Alert: Your Summer Wardrobe Must-Haves Inside!"

Both versions of the email contain similar content showcasing the new summer collection with vibrant images, clickable product thumbnails, and a prominent call-to-action button directing recipients to the online store.

Results and Surprise: After analyzing the results of the A/B test, FashionFusion is surprised to find that Version B with the subject line "Hot Deals Alert: Your Summer Wardrobe Must-Haves Inside!" significantly outperformed Version A. Despite initial assumptions that a straightforward subject line like Version A would be more effective, Version B resulted in a 25% increase in open rates and a 20% higher click-through rate.

The surprise stems from the realization that the more attention-grabbing and urgent language used in Version B resonated better with their audience, driving higher engagement and ultimately leading to more conversions. This unexpected outcome highlights the importance of testing different approaches and continuously refining email marketing strategies based on data-driven insights.

0 Upvotes
ChucksIhebuzor
Member

What’s an A/B test you’ve performed that surprised you and why?

I haven't done tests myself but have written email subject lines to be tested. I must say I was surprised by the insight from the lesson. The image with text got a higher CTR than an email with an Image and text. It shows the importance of A/B testing.

0 Upvotes
CLobo1
Contributor

What’s an A/B test you’ve performed that surprised you and why?

To determine the optimal message for a live webinar on business forecasting, we ran an A/B test on landing page content and emails driving to them.

 

Variation A – Positive connotation message “Enhance your Business Forecasting with Automation”

Variation B – Negative connotation message “Is your Forecasting process manual and error prone?”

 

Goal: While our target audience was Business Analysts and Financial Controllers in mid-size companies (we were aware of the possibilities of technology available for large organizations), the goal was to capture the sentiments that most attracts this cohort. Are the really excited about enhancing or respond to actions of fear.

 

Test Metric – To estimate that these two mutually exclusive variations are statistically significant at 95% confidence interval i.e. >5% P Value

 

Duration – These tests were live with the landing page, email and social media banners promoting the webinar on the same date/time but with different messaging.

 

Final outcome – Variation A was the winner with the test hypothesis showing the results as statistically significant from the Variation B responses. This led us to the conclusion that the word Automation was a key driver into capturing our audience attention given all the focus around automating and digital transformation in the office of finance.

chrlee
HubSpot Employee
HubSpot Employee

What’s an A/B test you’ve performed that surprised you and why?

Wow. I love the idea of changing your subject line's sentiment. Such a cool A/B test. 

CLobo1
Contributor

What’s an A/B test you’ve performed that surprised you and why?

Thanks, Christine. In addition to the response above, I would also like to add, now that we are able to identify words like automation gets the attention of our cohort, we are now testing the email copy to include two different messages. Around Automation and AI. We are also using tools like keywrds.ai and Magikstem to see what else comes up as popular keywords for our audience. 

KritikaSaluja
Member

What’s an A/B test you’ve performed that surprised you and why?

Hello, I'm Kritika. In a previous role, I conducted an A/B test for our sales email announcements. I collaborated with the Marketing and Creative teams to create two versions, A and B, each with a different color layout. After analyzing the results, we discovered that version B outperformed version A in terms of engagement and click-through rates. It was a pleasant surprise to find that our hypothesis was correct. Based on the successful performance of version B, we decided to use it for the email campaign.

SKomi
Member

What’s an A/B test you’ve performed that surprised you and why?

I have performed an A/B test as a produce clerk at the Jewel Osco that I work at. I had to help with an email to push sales on the fresh cut fruit. The two rivals were "The only way it would be fresher is if you grew it yourself!" The other one was "Most options for fresh cut you've ever seen at Jewel Osco!" The first one worked better and stuck as a marketing slogan for the produce department of Jewel Osco.

Valeriatma
Member

What’s an A/B test you’ve performed that surprised you and why?

Hi, I am Valeria, I performed an A/B test at a local company in my city to understand what gets the interest of their leads and after doing different projects, it is amazing how this helps to understand a segmentation of people and how we can analyze their behavior.

0 Upvotes
LNgqayimbana
Participant

What’s an A/B test you’ve performed that surprised you and why?

My name is Luvuyo i want to know, What makes an effective marketing email ? What are some things that should be included in one?

0 Upvotes
BBarrett138
Member

What’s an A/B test you’ve performed that surprised you and why?

I have not completed any A/B tests as I am still a student. However, I hope to do so when I enter the workforce. 

0 Upvotes
RGoyal57
Member

What’s an A/B test you’ve performed that surprised you and why?

 

I have not completed any A/B tests as I am still a student. However, I hope to do so when I enter the workforce. 

 
0 Upvotes
 
 
 
 
 
 
0 Upvotes
caligirl15
Member

What’s an A/B test you’ve performed that surprised you and why?

An A/B test I've performed in my previous job was when we had to send out different versions of announcements of a webinar my former company was hosting. We tested it first with the Marketing and Creative team using different colors for the email layout. Once we established that version B performed higher than version A, we went ahead and used version B. Surprsingly, our hypothesis was correct. Version B perfomed really well and we got a high engagement rate and click-through rate in our email.

0 Upvotes
iqrakhan
Member

What’s an A/B test you’ve performed that surprised you and why?

Not a marketing campaign per say, but we did AB testing on our headlines to measure the click through rate

MAbrahams5
Member

What’s an A/B test you’ve performed that surprised you and why?

Not a marketing campaign per say, but we did AB testing on our headlines to measure the click through rate

0 Upvotes