topic Re: contentsearch API returning nothing for private/membership blog in APIs & Integrations

contentsearch API returning nothing for private/membership blog

MichaelMa — Mon, 31 Jan 2022 15:20:07 GMT

Is there a setting/field that I need to add in order for private/membership required pages to show in the contentsearch api results?

I can retrieve the pages that are indexed via the get index properties:

https://legacydocs.hubspot.com/docs/methods/content/get-indexed-properties

So I know that the blog pages are being indexed. But when I do a search (yes, while logged in), the results show no results (total: 0).

Search works fine on non-private blogs.

Standard search query I'm using.

https://api.hubapi.com/contentsearch/v2/search?portalId=XXXXXX&groupId=YYYY&term=trends

Re: contentsearch API returning nothing for private/membership blog

dennisedson — Wed, 02 Feb 2022 19:01:26 GMT

@stefen 👋

Any help you can offer?

Re: contentsearch API returning nothing for private/membership blog

stefen — Wed, 02 Feb 2022 20:17:06 GMT

@dennisedson @MichaelMa as far as I know, the HubSpot crawler behaves exactly like a Google crawler in that it will only index public pages and also adheres to the robots.txt file.

That would be something you should post to the Ideas board and try to get some upvotes on. Technically it should be possible.

Re: contentsearch API returning nothing for private/membership blog

MichaelMa — Wed, 02 Feb 2022 20:25:25 GMT

The documentation under "Default indexing behavior":

https://developers.hubspot.com/docs/cms/features/content-search

CMS Membership restricted pages will only display in hubspot content search for users that are signed in and have access to the pages.

Which seems to indicate otherwise.

Re: contentsearch API returning nothing for private/membership blog

MichaelMa — Thu, 03 Feb 2022 13:36:33 GMT

So I'm not sure if I found THE solution but I found a solution.

Documentation for contentsearch API:

https://legacydocs.hubspot.com/docs/methods/content/search-for-content

Which basically shows using the endpoint:

https://api.hubapi.com/contentsearch/v2/search?portalId=62515&term=hubspot

However, one thing I noticed in the difference between the custom theme we paid hubspot to make and the marketplace theme we purchased was the end point. One of our blogs is public and uses the custom theme we paid hubspot to build. The other blog, which we're having issues with, is private/gated and uses the market place theme.

The end point hubspot theme uses is (replace subdomain/domain with your blog domain):

https://subdomain.domain.com/_hcms/search

eg, https://subdomain.domain.com/_hcms/search?portalId=62515&term=hubspot

Cloning the theme to a child theme and modifying the search endpoint to the same format as the one hubspot theme uses returns the same JSON response as the normal contentsearch API. It even converts the subdomain/domain combo into &domain= query variable found in the cachekey. More importantly, it works for private content if you're logged in! If you're logged out, it shows the same response as the normal endpoint (no content).

On a pure guess, I'm thinking that the reason the main endpoint doesn't show data is because of a Cross-Origin Resource Sharing (CORS) issue. Aka, the hubspot endpoint domain doesn't not have permissions to access the cookie data in the domain you set up for the blog.

Hope this helps anyone else having this issue. I know I searched for any direction for a few days before stumbling upon this "solution".