Protecting your website from AI Crawlers

[Anton]

Hey folks, 

doing some research and need your input.

When it comes to your HubSpot blog, web-/landing pages, how "protective" are you when it comes to Ai crawlers potentially crawl your post/page and use the content of it?

I know that membership/password protected pages are most likely not gonna be crawled as a crawler doesn't have credentials to access it, but what about regular pages?

Are you ok with Ai using your content or would you rather have the option to decide if an Ai should be able to crawl a page/specific area of the page?

best, 

Anton

[DanielleGriffin]

@Anton,

This is an interesting question.

I agree with @melindagreen that it would be nice to have the option to choose yes or no for allowing the crawlers to pull information or not.  

I honestly don't have a strong opinion for either side.  @DamienGriffin writes more of our blog content than I do (he writes 4 posts per week and I write 3 posts per week), so he may have a different take on it.  For my posts, I end up posting it in multiple other platforms anyways, so if an AI tool wants to grab it and train/use it, it wouldn't be hard for them to do so.  

[melindagreen]

IMO, options are always good. I can see some of my clients going "Yes, please put my content out there!" and others going "Nope, this is intellectual property." The option to share or not share is going to get more and more important—Cloudflare and some others opened that Pandora's Box, and I don't think we'll ever be able to completely go back. Options are a good thing.