Feb 27, 2020 9:50 AM
Recently in HubSpot analytics, there has been a huge jump in bot traffic to our website. Initally when this issue started, we went into settings and excluded bot traffic. Since, we've still seen a ton of bot traffic to our website. HubSpot did a bit of digging on the backend and saw it was coming from a few sources and asked if we could exclude (many) IP addresses. Is there a better way to solve this issue?
The longer we wait to sove this issue the more invaild our data becomes.
Feb 28, 2020 10:44 AM
Our website has also fallen victim to bots, especially in the past (2018 in particular, we had a ton of email bots). One area I would look at first is email. If your company does a lot of email marketing, I would check to see recent emails you've sent and if a lot of contacts are clicking the same links multiple times, clicking immediately upon receiving the email, or are clicking and then immediately bouncing. These are all possible signs of bot activity within a contact's email domain.
If you can identify some common themes with the email clicks, you may be able to filter out specific contacts whose emails are connected to bot activity (these contribute to a ton of Direct Traffic, rather than email traffic, so be sure to check on your metrics and see if that is happening).
A similar discussion about bots was brought up here before (linked here). My thoughts were the following, which looked into Google Analytics as well:
"Some people (myself included) noticed that HubSpot bots were causing some outrageous spikes in direct website traffic to their website home page. Since HubSpot filters out traffic from their own bots (at least as far as I can tell) those numbers won't appear in HubSpot's analytics. This could be why your home page is getting some crazy spikes in traffic in Google Analytics when compared to HubSpot.
We tried resolving this by turning off tracking for all bot traffic on our website, via Google Analytics. This means that, while bots will still go to your website, Google Analytics won't keep track of that traffic and won't factor it into your total traffic (Note: this does not change anything retroactively—only moving forward)."
Mar 2, 2020 9:48 AM
@Phil_DM2 Thanks for much for your response.
Unfortunately, the traffic source isn't coming from email. It's direct traffic. We're seeing spikes on a few orphan pages. Our challenges is those numbers aren't reflected on Google Analytics. So it's hard to solve for...
I've reached out to HubSpot support to export a list of IP addresses to see if I can exclude those IPs from our reporting.
However, I now have invalid numbers for the month of February because I thought excluding bot traffic would solve my issue. When it comes to YoY reporting our numbers are going to be thrown off. 😞
Mar 5, 2020 3:57 PM
Ahh, I see. Unfortunately, since you are unable to retroactively remove bot traffic from your analytics data, your numbers will be inaccurate for that particular month. This has happened to us before as well (what I refer to as "The Great Bot Debacle of 2018"), but one suggestion I have that could possibly help is to narrow down your analytics more for the month where bots were appearing.
Since bots are almost always going to be Direct Traffic (there are exceptions for sure, but typically Direct Traffic is most common since this basically means the bot's source cannot actually be tracked accurately), you can focus on each individual day in the month to see which had the biggest spikes in Direct Traffic. It is possible that this could be every day, but you may also find that only a select few days in the month had major bot traffic that skewed your numbers. You can then subtract those Direct Traffic numbers that you've identified as most likely to be bots, which might help your analytics report for that month.
This isn't a perfect solution of course, but it might help with your reporting for the month.
Mar 12, 2020 1:34 PM
@Phil_DM2 Thanks for this insight. I asked HubSpot for the IP addresses of the traffic to those pages. We found many are coming from the same source. I've reached out to company to see what is going on and if they can stop the massive amount of traffic. Still waiting to hear back...
Cloudflare does offer bot management which I was thinking about for a hot second. However, we aren't seeing this issue on Google Analytics. It's really unfortunate our HubSpot Analytics are so off. Now, we're unable to use these numbers for reporting. 😞