topic Re: How to noindex blog tags, categories and news archives in CMS Development

How to noindex blog tags, categories and news archives

srid2706 — Tue, 20 Mar 2018 10:39:53 GMT

Dear All,

My website analysis shows that google indexes the blog tags, categories and news archives of my sites. And this is creating multiple duplicate title, meta description and content issues.

Kindly let me know the way to noindex the blog tags, authors pages in Hubspot

Re: How to noindex blog tags, categories and news archives

AJLaPorte_diagr — Tue, 20 Mar 2018 14:42:23 GMT

You can always use the robots.txt file configuration in your Content Settings > Web Pages > Page Templates area. There you could disallow the crawling of the directories you would like. See here for more info on using the robots.txt file: https://moz.com/learn/seo/robotstxt

Another way to do it is to use HubL in the blog template header and check for tags listing pages / archives and then insert a meta tag for no-index no-follow into the header that way. Something like the following (I have not tested this but it should be close to this):

{% if is_listing_view %}
   {% if topic or content.archived %}
   <meta name="robots" content="noindex, nofollow">
   {% endif %}
{% endif %}

You can use the "Developer info" from the hubspot sprocket dropdown on the front end to find the exact variables you would need:

https://designers.hubspot.com/docs/hubl/how-to-use-developer-info-on-cos-pages

Hope this helps!

-AJ

Re: How to noindex blog tags, categories and news archives

ravenousblue — Mon, 12 Nov 2018 20:48:23 GMT

Can you guide me on how to add this to my blog header for https://www.cobblestonesoftware.com/blog/archive/2016/02

Do I need to update the code you shared?

Re: How to noindex blog tags, categories and news archives

AJLaPorte_diagr — Mon, 12 Nov 2018 21:08:40 GMT

Hi @ravenousblue,

Since this was posted, HubSpot has made some updates. If you go to your Settings > Website > Pages > SEO & Crawlers Tab you will see some settings for canonicals.

Under the "Pages and blog posts" area, choose:

- Add self-referencing canonical URLs to all pages and blog posts (Recommended)

Under the "Blog Listing Pages" if you choose the option of "Set each blog listing page’s canonical URL to the first page in its series" this will essentially stop Google from listing all the paginated pages in your site and instead each paginated listing will point to the first page in the series as the canonical.

There has been some chatter about these settings in the developer slack. Also, in a recent article from Search Engine Journal, there was talk about whether the duplicate content that is flagged is actually an issue. The article references this tweet exchange from Joose de Valk (of Yoast SEO WP Plugin) and John Mu (Webmaster Trends Analyst at Google)

Hopefully, this helps point you in the correct direction.

-AJ

-----------------------
AJ LaPorte
www.wsol.com

Re: How to noindex blog tags, categories and news archives

ravenousblue — Tue, 27 Nov 2018 20:29:44 GMT

Thanks AJ. I updated the blog settings accordingly.

Re: How to noindex blog tags, categories and news archives

AJLaPorte_diagr — Tue, 27 Nov 2018 20:31:34 GMT

No Problem! Let me know if you have any additional questions.

-AJ

Re: How to noindex blog tags, categories and news archives

grantfrontrow — Sat, 04 Apr 2020 20:46:53 GMT

is this even working? I updated it to "Set each blog listing page’s canonical URL to the first page in its series" on our blog and Google is still showing all the paginated pages / blog tag pages. Been a few weeks now since making change. For reference, the blog is: https://www.frontrowinsurance.com/articles