Nov 15, 2019 11:22 AM
Hi, HubSpot Community. I’m Jake from the HubSpot Product team. In the last year, the marketing industry as a whole has seen a spike in bot clicks in emails, so I wanted to take a moment to discuss what bot activity looks like, who’s most likely to be affected, and what you can do to address it.
What’s bot activity?
Bots are the source of a lot of internet traffic today. It’s estimated that more than half of web traffic originates from a bot, and bots are more sophisticated today than even a few years ago. Not all bots are bad bots, though! Google’s search engine bots, for example, crawl websites and index them; HubSpot’s chatbots help to facilitate conversations with your prospects and customers without requiring a human.
Unfortunately, there are also bots that create fake clicks and false impressions for apps, websites, and emails. With email, bots are designed to click links in emails as a way to explore, identify, and prevent links to malware or phishing attacks from entering a recipient's inbox.
Okay, so what does bot activity in emails look like?
Email bot activity differs from what bot activity may look like on a website. Bot activity in email is typically security software sitting in front of a recipient’s (or contact’s) inbox. That security software will open the email and, typically, engage with some (if not all) of the corresponding links in that email, following them through the redirect. These actions simulate real user engagement potentially causing false opens and clicks data to be recorded.
Every email marketer who has sent an email campaign has probably been affected by bots. And bots are tricky for email marketers because the email engagement events (opens and clicks) they generate can get recorded alongside legitimate ones. This leads to email campaign statistic inflation as well as automation being tripped based on these false events.
Who’s more likely to be impacted?
Bots don’t impact all marketers equally. These security filters are much more common in industries tied to finance or healthcare but aren’t as common in B2C marketing. Generally, regulated industries where contacts use a company email see more bot clicks than personal email addresses on email providers like Gmail or Office365.
So, what can we do about it?
For HubSpot, identifying bots and separating them from legitimate contact engagement has been, frankly, a game of Whac-a-Mole. In the last year, the industry as a whole has seen a spike in phishing emails. This rise in phishing has led to corresponding increases in companies adding security software (i.e., more bot activity).
At HubSpot, we’re doing our best to address this rise. Currently, we’re working on redesigns to email reporting to better separate legitimate contact email engagement from that generated by security software. The entire team at HubSpot, including myself, recognizes this can cause anxiety for a marketer. We will continue to do everything we can to find a solution that best addresses this problem. Until we have a full solution, we’ll be monitoring this thread to answer any relevant questions or concerns and respond to any feedback you have about this.
Solved! Go to Solution.
Feb 10, 2020 9:07 AM
Posting an update to this community thread as we are releasing some new bot detection software for marketing email. We have had a solution to bots in email for a while now. It was an IP based approach that worked fine but had some flaws. As we have dug into this problem it has become clear that increases in bot activity is an issue felt not just by HubSpot, but the entire email industry.
This week we are rolling out new tools for bot detection to HubSpot customers, and here are some notes on it:
It is impossible to capture all events generated from bots. If you believe you are still seeing bot activity feel free to post comments here or open a ticket with our support team. Our engineering team is continuing to monitor bot activity on our email and will take appropriate additional steps if they are needed.
Jun 21, 2022 10:55 AM
I did not implement this, it was only an idea I had. I would think you could implement this by having a hyperlinkink where the text is white on a white background. Humans wouldnt notice it and would not be likely to click it, but a clickbot would certainly notice it and click it. The idea would be that any emails where that link is clicked we can be reasonably sure it was a bot. The next issue is how to parse that info out in HubSpot which I am not sure is possible as of now
Jun 28, 2022 9:25 AM
We did implement and invisible image and labeled the URL as ".../honeypot."
That's been helpful to determine which companies are using bots. We also created a workflow that looks at the email click/open activity time stamps because we realize bots have predictable behavior in that the timestamps for delivered/open/click1,2,3,4 (however many buttons and hyperlinks you have regardless of whether they are HTML or plan text, including the honeypot) activities are all within the same minute of delivered or 2 minutes max.
We created a workflow to create an active list of any recipients who have opens or clicks one day after or longer after. We are using this as our actual engagement metrics and those are these people who get the next touch in the workflow for "responders. " It's a much smaller number of recipients than the clicks/opens that show up in the performance report, but it's at least more trustable/true/accurate. I'm going to post this workaround/trick at the end of this thread so all can see it - we are just starting to implement it with sends going out today. I'm curious what you think about this method and if you see any 'gotchas' - I really appreciate you responding to my reply, it's been feeling like the lonely wilderness out here trying to figure this out.
Mar 16, 2021 8:24 AM
Agree -- not sure with HubSpot isn't just productizing a work-around solution like that!
Feb 16, 2021 4:36 PM
Thank you -- any further updates or insights on these metrics and bot behaviors now that so many recipients have been working from home and not necessarily logging in via a VPN, instead relying on their home wifi IP?
Oct 1, 2020 1:16 PM
We have noticed similar patterns when sending marketing emails to goverment agencies specially airport administration and Department of Transportation (DOT). It appears all the members of the organizations open or click the email. In turn, this data gets reported under the email performance metrics. Does the new filter work with these types of accounts? This will help us have more accurate reporting of email performance and clicks.
May 25, 2020 12:05 PM
Thanks for rolling this feature out, but I just want to report I saw today a pattern of suspicious bot activity on one of our email sends. We were able to detect it because:
- a large percentage of recipients from the same email domain clicked on the links within a minute of receiving the email
- none of them converted (they had to fill a form to get access to the white paper our email was promoting), while the conversion rate at other companies was roughly 40%.
I opened a ticket with the support team, but you can email me if you want details.
May 28, 2020 9:16 AM
Sending you a direct message to follow up on that support ticket.
Feb 10, 2020 9:02 AM - edited Feb 10, 2020 2:48 PM
Jan 29, 2020 8:13 PM
Thanks for your post. I only saw this thread now. We noticied major issues last year regarding US traffic (we are Australian based). I flagged this with the support team who danced around the issue and did not really give me an good answer which I was quite disappointed with. I'm glad there is this thread acknowledging it.
I just wanted to post to flag this again as a key issue for our marketing reporting and metrics, because it largely skews the data. Hopefully a fix is discovered soon.
Jan 8, 2020 8:49 AM
Yes, they are . Can you please resolve this issue as we cannot distinguish between "clicks" and "opens" . This inability from Hubspot to fix this is having a direct impact on my ability to perform my job. I am imploring you to please give this a priority
Jan 6, 2020 2:35 PM
we are receiving "clicks" even without the recipient clicking the link at all. There is no distinction between opens and clicks , this has happened after applying secure mail gateway, we have opened a ticket and it was transfered to you if you have reached for a workaround/solution of the issue yet, your quick response is highly appreciated.
Dec 10, 2019 7:24 PM
This was also posted to the ideas forum.
One thing we've been doing and it's partially working (i.e. - need to send more email and this solution will never be a 100% answer as spam systems are always changing the way they work and no two are exactly alike), but add an invisible link to a page on your site in your emails. For example, in the below screenshot, "Communication" had a link to an old blog about "Communication", but it looks like all the rest of the text so unless you got lucky and moused over it, you'd never know it was a link. Then we segmented the people (aka bots) that were sent the email and clicked on the link and sure enough, by looking at the details of the clicks it's very clear it's a bot clicking (screenshot below - this is a monthly newsletter so has a ton of links in it). The next step which is still a work in progress is to add a specific list of domains that have suspicious / bot clicks and then segment these out of any automated follow up that is based on activity. One thing to mention is that some bots / spam systems will not click on every link in an email, but by doing this on every email you send, you can start to identify / weed out the domains that have aggressive spam systems in place that adversely affect reporting and other automated tasks. You can also segment your sends so that one email goes to suspicious domains and the other goes to people that don't have the suspicious activity. Then you can manually review the suspicious activity to determine next steps. Not a perfect solution and I'll share more on some things we're working on.
Dec 16, 2019 8:58 AM
Thanks for sharing @Mike_Donnelly! Very interesting solution, basically a captcha like solution for email. Mind sharing the ideas forum post here?
Dec 17, 2019 9:27 PM
Hey @jaleonard19 - Thanks and the above was everything I posted in the ideas forum. We're working on a few other ideas which will flag domains, but it's still a work in progress. Essentially, if you bubble up the top domains clicking a lot of links, you can be extremely confident they're coming from bots / spam systems. Then you can further investigate by looking at the timeline. Much of what you've specified is what we've been working on. Not a perfect solution (as it never will be), but it removes a lot of noise.
Dec 4, 2019 12:42 PM
Thanks for posting this. I notice that in my emails, 14% clicked on the logo which kind of makes no sense. If there is a beta or something, I would be interested!
Dec 9, 2019 4:35 PM
Hi @mrspabs - eeeekkkk. Yea that doesn't sound right, sorry you are running into that. Unfortunately, we are not doing any formal Beta, rather we are doing a generalized slow-roll out of the solution.
Nov 18, 2019 11:12 AM
Very interesting. Our company has noticed strange email open/click behavior in the past. Most notably, we had some unusual spikes in direct traffic from recipients only clicking the first link in every email we sent (the link happened to be from our company logo and led to our website's home page, which may have caused it to trigger as Direct?)
Essentially, we were seeing dozens of clicks from each email but with absolutely no engagement otherwise. It was really bizarre, and it did seem like some strange bot activity. Even this year, sometimes our emails consistently get the same number of clicks from the same people (give or take 20-23 clicks), while other emails receive nothing at all. Strange...
Nov 19, 2019 4:35 PM
Thank you for sharing. A lot of that does sound like bot activity. Two common patterns are:
Nov 25, 2019 5:24 PM
Oh wow, this totally lines up with what we've been seeing! Most notably, a contact clicks a link multiple times as well, and it's almost always as soon as the email is delivered to them. Is there anything we can do to avoid this or prevent it from continuing? Last month it completely skewed our metrics since we had way more Email traffic than we normally would. Now, we seem to be having a lot less.
Dec 2, 2019 10:32 AM
Hi @Phil_DM2 - Thanks for sharing. That is unfortunate that it skewed your metrics the previous month. We have been doing some interim work to identify some of the worst offenders and make sure those open & clicks aren't recorded. This may be why you are seeing less now.
In terms of what you can do - nothing at the moment. We are in the midst of rolling out a new solution for bots on email. So you should continue to see your metrics get more and more accurate over the coming months.
Dec 13, 2019 4:52 AM
Hi @jaleonard19 ,
When you say you're identifying the worst offenders and making sure their engagement isn't recorded, does that mean we won't get any engagement metrics at all from certain accounts? If so, I think that's a bit heavy handed.
Also, there's another obvious sign of a bot that hasn't been mentioned here: when a contact registers as both a click and a hard bounce. If illusory engagement like that could be filtered out of the metrics, rather than entire accounts, that'd be the most balanced solution IMO.
When the new solution rolls out, will there be a report of what filters have been implemented? Would be good to know when this happens so we can start using email engagement data for lead scoring and other purposes.