HubSpot Ideas

BW

Meta Noindex Blog Tags and Author Pages

It would be nice if you could easily noindex blog tag/topic pages and author pages.

 

There are 36 pages being indexed in google for our blog tags and authors. All 36 of these pages are thin content and showing up as duplicate content using siteliner.com which isn't helpding our visibility in search.

 

Our only recourse right now is to add /author/ and /topic/ to our robots.txt file. However, I doubt this fix is likely to remove them from Google's index permanently because there is a link to the author profile and topic page on every blog. Maybe editing the blog post template to unlink the author name and topic on blogs could help?

 

Either way, let's get this feature made HubSpot!

11 Replies
BW
Member

Edit:

 

You can use Google Tag Manager to accomplish this.

 

I followed this guide:https://organicdigital.co/blog/how-to-implement-robots-noindex-tag-with-google-tag-manager/

 

The DOM Ready events I used are "page URL contains /author/" for one tag and "page URL contains /topic/" for the other tag.

 

When setting up your tag, make sure the "Support document.write" checkbox is checked.

SimonS
Top Contributor | Elite Partner

Hi BW,

 

It's a great idea to have the ability to 'no index' pages. I would like to be able to do this on a per page base as well as per tag or author.

 

Great work around BTW, but it should be a feature instead 😉

 

Cheers, 

Simon

nicolerohrer
HubSpot Employee

I think it would also be helpful to have an option that there be no author or tag page created at all as a few customers have requested the ability to remove them entirely.

a_schumacher
HubSpot Product Team

Hi all! Thank you for your feedback here about wanting to no-index author and/or blog tag listing pages. Giving more granular control over which pages Google can/cannot index is something my team is reviewing. I have a few questions to help me better understand your needs/use case here:

1. Can you help me understand why the 36 blog tag/author pages being indexed is a negative (as opposed to a good thing with more pages being indexed/visible in search)? In my research, I have seen pros/cons for each so I would love to hear how you are thinking about it

2. Can you tell me more about how you are using blog author pages and category tags today such that there is significant overlap between authors/tags (enough that it is being surfaced as duplicate content)?

Thanks!
Best,

Amanda

Esther_ISM
Participant
Hi Amanda, I don't know if you've received messages with answers about questions. However, my answer on the first question would be as following. 1). My goal is not to have as many pages indexed as possible. My goal is to have only qualitative pages indexed. Qualitative as in pages that people search for and therefor I want to have it indexed. Having pages indexed that do not attract search visitors only means that I'm spilling crawling budget and link juice. Particularly in our situation this is the case, since we have 3 types of listing templates (for events, news and blogs/whitepapers etc) which each create their own author pages. Not all authors attract visitors from search but they could be having an author page because they host an event, another author page because they wrote a blog and another because of a newsitem concerning their product/service. Preferably I would like to have just one author page (although having several listing pages), and decide per author page if it should be indexed or not. Please reach out if you have some questions about my answers. Best, Esther
MariZanchetta
Contributor

This would be great. I didn't want to mess with the robots.txt file. Would be great if we could edit author, tag and other system pages, and be able to add the "noindex" in the HTML section just like we can do with the regular webpages and blog pages.

 

We also should have the ability to remove/unlink those pages from the blog posts. Otherwise, they are still being accessed anyways. 

 

Waiting to see when Hubspot will make this possible! Over 2,5 years since this topic was created.

@a_schumacher 

 

@BW thanks for the workaround! 

MSerranoSolà
Participant

Hello everyone!

 

Do you know if HubSpot has already worked on this topic? I also don't want to touch the robots.txt file, but I would really like to have all author and tags pages as noindex. However, I cannot find any place where to add this tag, as we can do with the blog pages.

 

Thank you!

Pixelwerx
Contributor

If it's helpful, one idea I just tried in my blog index template is this:

{% if tag or author %}
  {% require_head %}
    <meta name="robots" content="noindex">
  {% end_require_head %}
{% endif %}

Oliver

tko
Contributor

@a_schumacher Is there any progress here? We would like an option in the settings where I can set "noindex" for all tags and/or author pages.

SofieHermansen
Member

Yes, we are also missing this feature A LOT. With four languages. Ten different authors, and using the blog module for also resources and cases - which all have tags and authors - we have about 200 pages that are indexed but with duplicate and thin content 😥

 

Jake_Lett
Guide | Partner

You could also use your robots.txt file to restrict tags and use a wildcard
 https://knowledge.hubspot.com/seo/prevent-content-from-appearing-in-search-results#use-robot-txt-fil...

 

User-agent: *
Disallow: /topic/*
Disallow: /tag/*
Disallow: /author/*