There's an emerging standard called LLMs.txt which is basically like a sitemap but explicitly for LLMs. It would be great to have support for this in HubSpot CMS. While I could do it one time and upload the file to my site, really, it's best if it were being updated dynamically as my site changes over time. Today, I believe this HubSpot CMS feature lives under Content Hub.
Thank you all for your patience. While we continue to monitor llms.txt relevance, I wanted to resurface the following workaround that @katie previously shared:
If you're interested in generating an llms.txt file, there are some free tools available. Once you’ve generated the file, you can host it in your file manager. From there, you can set up a proxy redirect from /llms.txt to the location in your file manager where the file is hosted.
Thanks for the post! We've been following the llms.txt stuff closely as well. We've done a few deep dives on our web traffic data and as of now, we're not seeing enough of an indication that answer engines (ChatGPT, Claude, Perplexity) are using llms.txt as a convention for crawling web data. That being said, the idea definitely seems to be gathering steam, so while we are holding off on building a solution right now, we're going to keep very close tabs on this and are ready to build something if or when we see indicators that llms.txt are starting to be used to gather web data by answer engines.
In the meantime, if you're interested in generating an llms.txt file, there are some free tools out there, and once you have the file generated, you can host the file in your file manager. From there, you would set up a proxy redirect from /llms.txt to the location in your file manager that hosts your llms.txt file.
Hey there @SHilsman >> yes we will redirect to a file (it's not done yet, but we've validated that it'll technically work until HS provides the desired fix).
Thanks for the post! We've been following the llms.txt stuff closely as well. We've done a few deep dives on our web traffic data and as of now, we're not seeing enough of an indication that answer engines (ChatGPT, Claude, Perplexity) are using llms.txt as a convention for crawling web data. That being said, the idea definitely seems to be gathering steam, so while we are holding off on building a solution right now, we're going to keep very close tabs on this and are ready to build something if or when we see indicators that llms.txt are starting to be used to gather web data by answer engines.
In the meantime, if you're interested in generating an llms.txt file, there are some free tools out there, and once you have the file generated, you can host the file in your file manager. From there, you would set up a proxy redirect from /llms.txt to the location in your file manager that hosts your llms.txt file.
There is currently a proposal for standardisation to use an /llms.txt file on websites. This file would provide information to help LLMs like ChatGPT understand a website better: https://llmstxt.org/ .
With it, companies could attempt to tackle hallucinations/misinformation about their brand by providing the correct information in their LLMS.txt file.
HubSpot should add a section where this file can be created and managed, in the same way as a robots.txt file is.
The standard is proposing the file be hosted at the address: website.com/llms.txt
But if we were to create this file and upload it to the HubSpot file system right now, the URL would be website.com/hubfs/llms.txt with no way to edit the URL.
This is why a native solution needs to be developed.
We’re doing a lot of work around integrating AI into marketing strategies, and I see real value in this idea. Supporting LLMS.txt in HubSpot CMS would make a big difference for visibility across AI search engines, something that's only going to become more important.
On fileroom.com, for example, we’re always looking at ways to improve how our content is found and understood by both people and AI. This kind of functionality would help us, I'm more than confident that plenty of other teams like us would stay ahead of that shift.
Thanks for sharing this. Definitely worth supporting to help move it up the priority list! Silvia.
We also see requests to /llms.txt in our analytics data, and SEO tools such as SEMRush also check if a /llms.txt file is present. Manually creating and uploading the file in HS File Manager is not an option, as our website is constantly changing and the file would quickly become outdated. We therefore need a dynamic solution in HubSpot.
BUT: Is it not just crawled like any other file? E. g. did you place it under yoursite.com/llms.txt, with no links to from anywhere - or did you include a link to it anywhere?
