Are Bots Affecting Your Email?

[jaleonard19]

Hi, HubSpot Community. I’m Jake from the HubSpot Product team. In the last year, the marketing industry as a whole has seen a spike in bot clicks in emails, so I wanted to take a moment to discuss what bot activity looks like, who’s most likely to be affected, and what you can do to address it.

What’s bot activity?

Bots are the source of a lot of internet traffic today. It’s estimated that more than half of web traffic originates from a bot, and bots are more sophisticated today than even a few years ago. Not all bots are bad bots, though! Google’s search engine bots, for example, crawl websites and index them; HubSpot’s chatbots help to facilitate conversations with your prospects and customers without requiring a human. 

Unfortunately, there are also bots that create fake clicks and false impressions for apps, websites, and emails. With email, bots are designed to click links in emails as a way to explore, identify, and prevent links to malware or phishing attacks from entering a recipient's inbox.

Okay, so what does bot activity in emails look like?

Email bot activity differs from what bot activity may look like on a website. Bot activity in email is typically security software sitting in front of a recipient’s (or contact’s) inbox. That security software will open the email and, typically, engage with some (if not all) of the corresponding links in that email, following them through the redirect. These actions simulate real user engagement potentially causing false opens and clicks data to be recorded.

Every email marketer who has sent an email campaign has probably been affected by bots. And bots are tricky for email marketers because the email engagement events (opens and clicks) they generate can get recorded alongside legitimate ones. This leads to email campaign statistic inflation as well as automation being tripped based on these false events.

Who’s more likely to be impacted? 

Bots don’t impact all marketers equally. These security filters are much more common in industries tied to finance or healthcare but aren’t as common in B2C marketing. Generally, regulated industries where contacts use a company email see more bot clicks than personal email addresses on email providers like Gmail or Office365.

So, what can we do about it?

For HubSpot, identifying bots and separating them from legitimate contact engagement has been, frankly, a game of Whac-a-Mole. In the last year, the industry as a whole has seen a spike in phishing emails. This rise in phishing has led to corresponding increases in companies adding security software (i.e., more bot activity). 

At HubSpot, we’re doing our best to address this rise. Currently, we’re working on redesigns to email reporting to better separate legitimate contact email engagement from that generated by security software. The entire team at HubSpot, including myself, recognizes this can cause anxiety for a marketer. We will continue to do everything we can to find a solution that best addresses this problem. Until we have a full solution, we’ll be monitoring this thread to answer any relevant questions or concerns and respond to any feedback you have about this.

[TMerriam9]

Jake, please move this one back to 'Unsolved'. It's definitely still an active unresolved issue. 

Here's a really reliable methodology to identify domains using security software that actively and routinely clicks links:

• Any domain that (in a single email):
• Gets more than 1 click
• And whose CTR is higher than 50%
• Bonus points if you see that all of the clicks from the domain come within a few minutes of each other.

Please give us the option to filter out this activity. This week my email clicks were inflated to 3x reality because of this. I've just stopped emailing people who work for companies with this type of security parameter. But this week 3 new ones started doing it.

[RDean7]

This seems to have gotten worse. And it's easy for me to see bot activity, so I'm not sure why it's so hard for Hubspot to resolve the majority of these. If an email is sent, then opend and all links clicked in the same minute, it's clearly a bot. Espacilly if his behavior repeats for all contacts at a company.

While Hubspot does not show down to the second, I bet we would see opens and clicks happening within a few seconds. Also clearly bot activity that should be easy to filter out.

At this point I'd like to see more libral enforcemnt of scrubbing bot clicks and missing some real activity than being consrvative and having SOOO many clearly bot activty mess up all my metrics. Eepecially open-to-click rate, which, due to bots sometimes shows in the 80% range, but is completly bogus.

Do better Hubspot. Pardot and others have....

[AFeiner]

From my understanding, HubSpot groups Bot activity into Account and Behavioral-based activity. HubSpot is aware of Accounts where Bot activity is prevalent and behaviors where the timestamp of the email and the corresponding bot activity happen at nearly the same time. My interest is in the latter and we are hopeful that this type of behavior can be removed from our email results.

[vanessa2]

Bot filtering setting is not helping.

Another data point is corporate email servers will open the same email for all its recipients at the same time. For example, if you send to 3 people at the same domain, we see all 3 people open the email in the same minute. It may be well after HubSpot sent the email…but the opens and clicks are all done at once.

[SMassey]

We've been noticing issues with HubSpot tracking for a while now and when I submit a support ticket I just get the run around with no answers or solutions. Wasn't until today that I was finally told about the bot activity issues. But with us it's not just affecting our marketing emails, it's also an issue with one-to-one sales emails and landing pages. We're getting false notifications for email opens, clicks, page visits, and form submissions. 

Anyone else having the same issues when using pretty much any tool within HubSpot Marketing? I'm having a hard time justifying user costs given the data and analytics is a complete joke. 

[MMunroe]

Has anyone been able to come up with an all encompassing way, through either a list or a workflow, to even identify the folks and/or companies this consistently happens with? Something to the extent of if every link in Email A is clicked flagged this contact engagement as potential bot activity?

The only way I can see to do that at the moment would be to take every single email and say if opened and clicked on this AND this AND this... which obviously is not feasible to do for dozens of emails. 

Curious if there is a better way or even a condition that could be added by Hubspot in the list builder to say if All Links clicked in Email or if Links clicked at the same time email was sent... 

Would love to hear if anyone has come up with a creative workaround! 
Thanks

[JPullan]

Hi @MMunroe , not something I've implemented myself but saw this and think it mught work: https://sponge.io/6-ways-sponge-recommends-handling-bot-clicks-marketo/

It's for marketo but can't see why adding a hidden link to catch bot emails engagement and using a workflow to update a checkbox wouldn't work in HubSpot. You'd then need to use a smart list to find the 'true' engagement for each email. Sadly any of HubSpot's out the box reporting like Email Health would still be useless

[amyleopold]

Any updates on this? I just put in a support ticket for this as we're seeing an increase in bot activity that opens and clicks on our emails rendering our emails stats suspect.

Thanks!

[JPullan]

I'm getting a little concerned for HubSpot here. They do a fantastic job at making email engagement analysis very easy and powerful, with great reporting layout and tools to make performance comparision. However, the currency that makes this machine run is the engagement data itself and it's becoming increasingly unreliable.

On a call with multiple HubSpot employees we were not able to find a single email metric: opens, clicks, web page views, even email read time that was reliable enough for us to base behavioural triggers or reporting on. Our stats are so inflated we have little idea of how we are really performing and so instead of being enabled, we are really flying data blind - our instruments are giving us false readings. As a marketer that's a scary thing, we may think that our email content is really hitting home, where in fact there's been little real engagement at all. We may pass on 'hot' leads onto Sales for them to lose confidence in what we are sending their way as all this 'engagement' amounts to 0 new conversations'.

A good example of how this fake data erodes HubSpot's platform is the Email Health tool. A real value add that allows you to review your entire email marketing operation with a few clicks. However, it's completely useless for us because as all the building blocks (engagement data) going into it are inflated, the results we pull out cannot be trusted. 

As we work in B2B financial services we are facing the highest email security, so others in the post may not have the issue to the same degree, but given HubSpot as a platform is built on identifying engagement and enabling smarter marketing, if this fundamental issue isn't tackled the value of the platform as a whole will diminish, making it harder to justify the invesment in it. Even for those who are real fans.  

[Maen_Hadid]

Hi @jaleonard19 ,

We're suffering from the same issue, clicks are becoming unreliable metric to count on for qualifying & scoring leads.

Something in particular we noticed that wasn't mentioned here before is Plain text clicks.

Our logo link (1st link usually in our emails) is recording more clicks than all other links combined which obviously doesn't make sense, so upon checking we noticed that HTML clicks are normal (a bit on logo then majority on the rest, banners, CTAs...etc.) However, when checking Plain text clicks, we noticed that in most cases there's only our logo in that section with significant amount of clicks. Our bot filtering feature is activated yet we still see that.

For example: one email recorded 411 clicks on our logo link, 390 of those are under Plain text. Another emial recorded 375 clicks on our logo link, 359 under Plain text. The more recent we check, the worse the trend gets.

Across 2020, plain text clicks made almost 50% of our total clicks on average. Last half of 2020 the average got over 60%.

Do you think this is also an indicator of bot clicks? If so, is there any way we can build a Workflow that only qualifies leads who clicked on any link except our logo (which we can use as a ghost link, thanks @Mike_Donnelly !) after 5 or more minutes of receiving the email. (assuming most bots are clicking immediately upon receving)

Do you think this would help us reduce the amount of leads that gain false score?

Cheers,

Maen

[INGA_CG]

Hi Maen, 

I have dealt with this on several occasions now. It has been tied to schools and business email addresses in my case. As I dug into this, I connected the culprit email addresses to the use of Microsoft Azure & AWS. This is extraneous but interesting information, in my opinion.

As I've been brushing up on this topic and trying to get more answers, I came across these two articles that I felt were good resources to help explain the issue to my superiors. 

Barracuda: https://assets.barracuda.com/assets/docs/dms/Bot_Attacks_report_vol1_EN.pdf

eSputnik: https://esputnik.com/en/blog/how-to-detect-bots-case-study/  

Unfortunately, even with all of the research I have currently done, I still do not feel confident in explaining the behavior. Like you, I, too, have seen the inflation of clicks centered around the first URL presented in the plain text version of the email, but I have not found any blogs or whitepapers that call out the plain text version specifically. 

Now, I have found a solution to cut down on both the inflated clicks and the subsequent inflated sessions, but that doesn't explain why this started or what action I should take regarding the culprit's email account. I have assumed that security bots cause this behavior. Therefore I shouldn't remove the contacts as the end-user will have an opportunity to see the email after internal firewalls complete the spam checks, but is this an accurate assumption? Is continuing to email these contacts detrimental to deliverability and list health? Why does an issue like this all of a sudden occur? Also, why is the behavior not 100% consistent from one campaign send to the next? I would think that all email addresses associated with a domain that has shown this behavior would display the same behavior, but that has not appeared to be the case for us. Lots of puzzling questions, but here is one for you. Have you been able to gather any more details about your scenario? 

On a different note, my solution to combat this issue has been to make the first clickable link a "View email in browser" URL. Now, I use an alternate ESP and do not know if this will be a viable solution for you on Hubspot, but I figured it was worth sharing. The ESP I am on doesn't count clicks associated with "View email in browser" URLs; therefore, the reporting and list segmentation issues have cleared up. Additionally, since the first URL is no longer a page on our eCommerce website, the inflated GA session counts we were experiencing have significantly reduced volume (I've only implemented the above solution in blast/batch emails). 

Look forward to hearing from you or anyone else that may have more insight. 

- Inga

[Maen_Hadid]

Hey Inga,

Thank you so much for such a detailed reply, this is helpful and I feel
your frustration.

We actually moved to Marketo and now I'm just looking at the numbers as we
send out our first few emails using our dedicated IP. It seems like things
are going well, I would assume the dedicate IP would help a bit from a
sender reputation standpoint. I don't see super high open rate or click
rate so hopefully problem solved.

The ESP "open in email browser" trick is really smart though, I'll keep it
in mind for future.

Thanks again,
Maen

[anandkjain]

Hi,

I was shocked to see Hubspot reporting 70% open rate within a few seconds of sending the mail and all of them were opened by the same browser at the same time.

This was especially surprising because you would not find so many people sitting in front of computer at 4.45 am. Please check this bug.

[jaleonard19]

Hi @anandkjain,

Do you have bot filtering turned on in your HubSpot account? 

If you do and are still having issues then go ahead and fill out a support ticket. The issue does not originate in HubSpot's reporting but from recipients inboxes. It is bots (google image cache, security software, etc.) that are actually loading the embedded pixel we use to track email opens. Still, we work to keep your data as clean as possible.

Cheers,

Jake

[JPullan]

Hi Jake,

I've had a support ticket open for this for quite a while, having escalated it through our Account Manager, and the response is that nothing can be done except the improvements being gradually made.

Working in regulated B2B spaces our email engagement reporting, and any automation that relies on it, is currently useless. With over 50% of our 'engagement' being false we've had to strip opens and clicks out of Lead Scoring and rely on page views (to my knowledge these are still reliable, but we will lose anybody denying/blocking cookies).

Needless to say this is significantly denting the usefulness of a significant part of HubSpot's 'Marketing Hub', I am suprised that more of HubSpot's B2B customers aren't up in arms about this. Without any mechanism to report and improve the bot filtering, and no transparency over the timeline or existence of any planned changes, this must be weighing heavily on the renewal decisions of quite a few customers.

[darshakparmar]

In a B2B marketing environment, Click Rate has become the most unreliable matric. If Hubspot can find a way to exclude data of those clicks which happen within 60 seconds from the time email is sent, I think the majority of fake data will be eliminated from the report. This is the simplest thing HubSpot can do. 

[MikeW1]

Hello

Thanks for the update ..I  must say that this issue  is  the single most irritating and frustrating problem I am encountering. It prevents us from seeing accurate data. Too bad  you can't resolve it 

[jaleonard19 · Accepted solution]

Posting an update to this community thread as we are releasing some new bot detection software for marketing email. We have had a solution to bots in email for a while now. It was an IP based approach that worked fine but had some flaws. As we have dug into this problem it has become clear that increases in bot activity is an issue felt not just by HubSpot, but the entire email industry. 

This week we are rolling out new tools for bot detection to HubSpot customers, and here are some notes on it:

• The new tools take a behavioral-based approach. We look for patterns in the open or click events that indicate a likelihood of being generated from a bot.  
• We will be rolling this out to all HubSpot accounts this week.
• No action is needed. If bot filtering is turned on for your account then it will just work. 
• This does not change any email campaigns sent in the past. It only impacts emails going forward.

It is impossible to capture all events generated from bots. If you believe you are still seeing bot activity feel free to post comments here or open a ticket with our support team. Our engineering team is continuing to monitor bot activity on our email and will take appropriate additional steps if they are needed.

[drosenbaum]

Hello! I think a super effective solution here would be to do the following:

1. Add an invisible clickable hyperlink in the email somewhere that only a bot would recognize and click

2. Be able to filter email analytics to prevent clicks being tracked on any individual email send that has that unique link clicked. This would prevent it from adding click history to contacts, and exclude the numbers there from being included in analytics

[TSweeney2]

hi drosenbaum, i'm followwing this topic feverishly to improve our analytics. we have the bot filter turned on. i'm curious how you did this:

1. Add an invisible clickable hyperlink in the email somewhere that only a bot would recognize and click