Robots txt is set to Disallow

[AVision]

My blog is recently unable to read by google

https://adventure-vision-treks-and-travels-46145676.hubspotpagebuilder.com/

the default value in robots.txt is as follow

User-agent: *
Disallow: /

 while I set as following in Setting > Content > Pages >SEO & Crawelers

its not being reflected on live and my website is unable to be indexed on google.

is the are changes on hubspot free pages policy?

[FarioConsulting · Accepted solution]

Hi,

You will want to ensure that your settings override the default ones for a specific domain....you would need to click "Override default settings" after selecting the domain whose robots.txt file you wish to edit

Cheers,
Harsh
Fario Consulting

[Anton · Accepted solution]

Hey @KTaksi, 

as far as I remeber, hubspotpagebuilder.com domains are excluded from crawling.

In order to get your page crawled, you have to connect your domain to HubSpot. 

Best, Anton

[KTaksi]

Looks like no solution on this topic 😒

[Anton · Accepted solution]

Hey @KTaksi, 

as far as I remeber, hubspotpagebuilder.com domains are excluded from crawling.

In order to get your page crawled, you have to connect your domain to HubSpot. 

Best, Anton

[BérangèreL]

Hi @KTaksi and welcome, it's a pleasure to have you here!

Thanks for reaching out to the HubSpot Community!

Can you please share the steps you have already taken to have the robots txt set to disallow?

The more info, screenshots (without sensitive/confidential information), and details you can provide, the better the Community can assist.

Here are some resources for reference:

- Content Search
- Website Settings

Let's invite our Top Experts to this discussion: Hi @RidhiSharma2000, @Christensen and @Anton do you have suggestions to help @KTaksi, please?

Thanks so much for your help and have a fabulous day! 🦌
Bérangère

[Christensen · Accepted solution]

@AVision this are not "your" domain this are HubSpot System Domain which are meant to be staging domains. You can't also override the the default settings in Content > Pages > SEO & Crawlers for the System Domain. If it's crawlable previously then HubSpot must have rolled out an update and this means that you'll need to connect your own domain to fully use the HubSpot for your blogs and be visible to crawlers. If you are happy without search engine presence then you can continue to use the System Domain as is else connecting your own domain will be your only option.

[Maxim1]

I have a similar problem on the site https://alterpex-48328568.hubspotpagebuilder.com/.

The problem appeared recently; previously, the site was indexed. 

Account: free. 

I contacted support, they advised me to ask a question in the community

[FarioConsulting · Accepted solution]

Hi,

You will want to ensure that your settings override the default ones for a specific domain....you would need to click "Override default settings" after selecting the domain whose robots.txt file you wish to edit

Cheers,
Harsh
Fario Consulting

[AVision]

on setting > Content > Page > Domain
there this feature is disabled,

[kennedyp]

Thanks for the help so far, @FarioConsulting! Any insight regarding the updated information from @AVision? 😊

[AVision]

Where can I change this setting?? Override default settings?? I can not find it on dashbord, can you guide me the steps

[DianaGomez]

Hi @AVision,

Thank you for reaching out to the Community!

I wanted to tag in a couple of subject matter experts to see if they have any input on this matter:

Hi @alyssamwilie , @Lucila-Andimol, @BarryGrennan, @JuanGomez0818  do you have any advice for @AVision? Thank you!

Cheers
Diana, Community Team