Sensitive files attachment which is added to File Manager can be indexed and crawled
Today I found out that any file attachments sent via Gmail, with a log feature "on" or sent via "send later" of the HS plugin, will be added to HS File Manager - which can be indexed or crawled by Search Engine.
This can pose a serious issue especially when we're communicating with our clients and that we attach any sensitive files and those files are added to the File Manager. Since HS Support suggested that our content such as whitepaper or guide be stored on a folder with a No follow and No index tag on them via the robot txt, I'd conclude that any files not stored in that folder will have the potential to be found via search engine.
I highly suggest that HS add a feature where any files attachment added this way will be stored in their server without search engines being able to crawl or index them. This is a high risk issue that can violate the GDPR. Either that or I'm wrong in my understanding on how the File Manager works within HubSpot.
I found this out today as well, and I'm extremely concerned about it. Basically, any user that adds/uploads an attachment to a note or email in a contact record essentially places the file in the main directory of our file manager by default. Not only does this create organizational issues in our file manager because now we have a bunch of random files in the main directory, but more concerning is that this document becomes crawlable by search engines.
Some of those documents are indeed sensitive, and this puts our company and customer at serious risk and can violate the GDPR and other privacy regulations. The user does not have an option to specify where they want to upload these documents.
I suggest to the HubSpot team that any uploaded files associated with contact records go into a protected/secure folder by default - one that is now crawlable or publicly exposed. Changing the URL for uploaded files or adding to the robots.txt file is not a viable option, as we have no control over when users upload (or know about it). I think if other HubSpot customers knew about this there would be a tremendous sense of urgency around addressing it.
Thanks for enforcing this idea, langalich! I'm actually curious if any of HubSpot customers are even aware of this issue because if they are, I'm 100% sure they will be highly concerned about this.
Hi @dhika - I agree and sent a chat and an email to Hubspot support as well, expressing the urgency around this. Thank you for raising the issue first. I was glad to see we weren't the only ones seeing this.
Thanks for raising this concern! It is true that the emails you log are stored in your HubSpot portal and files attached to those emails are stored in the Files feature of your portal (File Manager). However, those files are not indexed by Google or otherwise discoverable for several reasons.
Files in email attachments are written to File Manager in hidden folders. That means you can see attachments when you’re looking at a contact record, but you can’t browse through them in the File Manager interface. If you want to view attachments to Sales emails you sent today, you need to login to your HubSpot account in order to view them. The links to the files’ locations also include a special string (a unique hash) that prevents them from being guessed, even by logged-in users.
To be triply sure that attachments aren’t indexed in search engines, we currently send instructions to Google and others to tell them to not index those files. More specifically, a X-Robots-Tag header is included in responses for the files in Sales emails, which prevents them from being discoverable in search engines.
So overall, today, you need to be logged in to view email attachments. And for files that might have been accessible with just the link, there are several other protections to keep them from being found accidentally.
If you have specific concerns about your data and the way it is protected, please contact the HubSpot Support team or your Customer Success Manager to get connected to the people who can best help address those concerns. Thanks again!
@rstinsonThanks for your thorough response. Good to know that now HubSpot took extra measures to protect these files.
Now just to clarify a couple of points:
Does this only include files as an attachment sent via HubSpot portal aka through the contact record, or also include emails sent via Gmail using HubSpot plugin?
Do the instructions you meant also include all files stored in the file manager, or only those that are added via first bullet above? Meaning, if I add a file manually to a file manager, would they be indexed by search engine?
Hi all! I just find out about this issue! This is a real problem! @dhika@rstinson@langalich What workaround did you find? I have to change the visibility for 3000+ files. Doing it manually doesn't look like a good solution.
How are you currently managing it? Do you regularly go over each file to edit the visibility property (default setting being "ON")?
I too just found out about this issue yesterday. Contrary to the HS response above, I found confidential files that had never been directly uploaded to the file manager indexed and searchable with public-facing URLs. These confidential sales documents being open to the public is astonishing and makes me question everything about Hubspot as a product and absolutely makes me question my sales teams use of the product. Is there truly no database segmentation between sales and marketing?
Beware, your confidential HS files your sales team might be sending could just be one bug or one product update away from being indexed by google and open to the world. I can say for me, that was beyond concerning. The fix, as others found, was to manually go through every file in the file manager and delete (I did not change their visibility as I wasn't willing to wait for them to be crawled to be removed and I didn't want them still accessible) one file at a time. It was painful, it was tedious and it made me question my investment in HS through every single one.
And on the marketing side, remember that every file you upload into the file manager will be indexable and searchable by default so keep that in mind as you add files or resources to pages. There is no indication of this at upload -- you have to go back in to change its visibility (and no just checking the tell search engines not to follow this link doesn't make it private). Oh, and if you delete or archive pages, those resources will still be available in the public and indexed. That makes maintenance on this platform tedious and frankly not worth the investment, IMO. Who is it really benefitting to have every uploaded file publically searchable by default with a HS URL? It doesn't appear to the customer (me), but instead, seems to squarely tick a few boxes for HS for how many URLs they manage or some other foolish metric. This was an alarming find and makes me think this tool is definitely not what it is cracked up to be.
I really hope we see a clear roadmap discussion on how they will update this to regain trust in use of this for sales and marketing.
Thank you for sharing your feedback and concerns with us. We recently went through and audited all of the places in the product where users could upload files. Through that process, we modified the default permissions to be as restrictive as possible based on the use case. We configured the files' visibility settings based on your anticipated objectives. In addition, we tightened the permissions on some sets of existing files to better meet user expectations.
In light of that, I wanted to provide some clarity around our files tool functionality and our file visibility settings as they stand now. Currently, there are three types of visibility settings (Public, Public: No Index, and Private). Even for files that are Public, it’s important to note that the files are not indexed by search engines or discoverable unless you take further action to add them to your website (such as linking them to a webpage or adding them to the sitemap.xml).
Here are more details on the three types of visibility settings:
Public: The file URL will be publicly accessible by anyone who has or guesses the URL web address. Search engines will be able to index the file URL only.
Public - No-index: The file URL will be publicly accessible by anyone who has or guesses the URL web address. Search engines are instructed not to index the file URL.
Private: The file URL will be private and requires a temporary URL to access. Search engines will NOT be able to index the file URL.
When you upload files, we now provide a notification about the file URL visibility type upon upload to the files dashboard and pickers so you’ll know if your file is Public. And for files that are Public (deleted or otherwise), the user would have to take action to add files to their website for the files to be indexed (though it’s worth noting that it is possible for someone to access a Public file by correctly guessing the Public file’s URL).
Also you mentioned needing to manually go through every file in the file manager to change visibility settings or delete. Thank you for highlighting that pain point, as we certainly understand that frustration. We have introduced the functionality to change the visibility of the files in bulk, as well as move, delete, and export multiple files or folders at once: https://knowledge.hubspot.com/cos-general/organize-edit-and-delete-files#edit-the-file-visibility-se.... Hopefully that addresses your needs there.
As a HubSpot user, I understand that management of your files and brand assets is essential to your work. That’s why HubSpot strives to give you control when managing your files within your portal with bulk management of file URL visibility and updated default settings. Our product roadmap currently has no plans to make files more public than they already are. Instead, other product updates already available or on the horizon include improved file search and more granular files tool user permissions.
The security and privacy of your files remain our top priority. If you have additional concerns about your files, please feel free to contact the HubSpot Support team or your Customer Success Manager who’d be happy to help you further. Thank you!