contentsearch API returning nothing for private/membership blog

[MichaelMa]

Is there a setting/field that I need to add in order for private/membership required pages to show in the contentsearch api results?

I can retrieve the pages that are indexed via the get index properties:

https://legacydocs.hubspot.com/docs/methods/content/get-indexed-properties

So I know that the blog pages are being indexed. But when I do a search (yes, while logged in), the results show no results (total: 0).

Search works fine on non-private blogs.

Standard search query I'm using.

https://api.hubapi.com/contentsearch/v2/search?portalId=XXXXXX&groupId=YYYY&term=trends

[MichaelMa · Accepted solution]

So I'm not sure if I found THE solution but I found a solution.

Documentation for contentsearch API:

https://legacydocs.hubspot.com/docs/methods/content/search-for-content

Which basically shows using the endpoint:

https://api.hubapi.com/contentsearch/v2/search?portalId=62515&term=hubspot

However, one thing I noticed in the difference between the custom theme we paid hubspot to make and the marketplace theme we purchased was the end point. One of our blogs is public and uses the custom theme we paid hubspot to build. The other blog, which we're having issues with, is private/gated and uses the market place theme. 

The end point hubspot theme uses is (replace subdomain/domain with your blog domain):

https://subdomain.domain.com/_hcms/search

eg, https://subdomain.domain.com/_hcms/search?portalId=62515&term=hubspot

Cloning the theme to a child theme and modifying the search endpoint to the same format as the one hubspot theme uses returns the same JSON response as the normal contentsearch API. It even converts the subdomain/domain combo into &domain= query variable found in the cachekey. More importantly, it works for private content if you're logged in! If you're logged out, it shows the same response as the normal endpoint (no content).

On a pure guess, I'm thinking that the reason the main endpoint doesn't show data is because of a Cross-Origin Resource Sharing (CORS) issue. Aka, the hubspot endpoint domain doesn't not have permissions to access the cookie data in the domain you set up for the blog.

Hope this helps anyone else having this issue. I know I searched for any direction for a few days before stumbling upon this "solution".

[stefen]

@dennisedson @MichaelMa as far as I know, the HubSpot crawler behaves exactly like a Google crawler in that it will only index public pages and also adheres to the robots.txt file.

That would be something you should post to the Ideas board and try to get some upvotes on. Technically it should be possible.

[MichaelMa]

The documentation under "Default indexing behavior":

https://developers.hubspot.com/docs/cms/features/content-search

CMS Membership restricted pages will only display in hubspot content search for users that are signed in and have access to the pages.

Which seems to indicate otherwise. 

[dennisedson]

@stefen 👋

Any help you can offer?