Our organization uses HubSpot across multiple international businesses and we leverage HubSpot’s cross domain tracking feature. This feature works well for users. It does not work well for crawlers.
An example of this issue is observed in the footer of https://www.pesi.co.uk
These links appear across all pages, and each time GoogleBot crawls a page, a new link is generated due to value changes in parameters like __hssc, __hsfp and more.
Example - 1
https://www.evergreencertifications.com/uk/?utm_source=pesi-uk&utm_medium=referral&utm_campaign=footer-partner-link**&__hstc=135057326.2f3f33a24b44870ec4a577029c49e44b.1727740800300.1727740800301.1727740800302.1&__hssc=135057326.1.1727740800303&__hsfp=1888902605**
Example 2
The result is an endless queue of URLs that GoogleBot will attempt to crawl as shown
This is a huge waste of crawl budget across multiple sites that needs to be addressed by HubSpot. Blocking cross domain tracking parameters in robots.txt will help, but it doesn’t resolve the issue and it also creates other issues.
An example of other issues is indexing. Google has choosen to index many pages with HubSpot tracking parameters and it has disregarded the canonical tag.
If I block cross domain tracking parameters in robots.txt, GoogleBot cannot see the proper canonical URL and it also cannot see meta tags like noindex if those were applied.
I purpose that their needs to be a change to HubSpot cross domain tracking parameters based on user agent. Cross domain parameters should not be applied to specific user agents like GoogleBot, BingBot and more like MSNBot.
I’m also open to other suggestions if there are other options to consider.