[Discussion] Sunsetting the Keywords Tool in 2018

Marketing & Content Hub Content Strategy & SEO
1

Hi All,

Jeffrey from the product marketing team at HubSpot here. We’re planning to sunset the Keywords tool within HubSpot this year, and I wanted to start a discussion here about why and answer any of your questions.

For a full timeline and background information, I recommend reading this post.

That said, here are 3 high-level reasons we’re sunsetting the Keywords tool:

  1. Search has moved beyond just Keywords. Search engines now base results on user intent and topics, and our exact match keywords tool was not ideal for telling you what to write about, or how you were doing.
  2. Rank is not a proxy for success. Many of the marketers that I speak with still want to “rank better” or “rank 1” for their content. While those aspirations are commendable, the fact is there is no consistency in search and back in 2011 Adam Lesnik, a Google search evangelist at the time, said that “rank checking is largely overrated and a gross waste of time.” It’s clear based on the personalization of search by localization, individual, device, and more than rank is becoming an irrelevant metric and is not a measure of success. Instead, we believe you should focus on attracting actual visitors, leads, and customers.
  3. Content Strategy is a new SEO tool that can help set you up for success. The topic cluster model works. We’ve seen a remarkable increase in traffic at HubSpot, and customers like IDS Agency, Townsend Security, and others have also experienced remarkable success. This tool not only helps you plan what to write, but also helps structure content in a way that is optimized for search engines and humans alike, and gives you insight into how many visitors, leads, and customers you are generating.

With that said, let me know if you have any questions or concerns. I’ll try to hop into this thread throughout the next few days to answer questions.

2

Hi @Jeffrey

Thanks for inviting a discussion on this.

While I understand the challenges HubSpot has had with the keyword tool, and I agree that search has changed dramatically in favour of localised and personalised results, I’m sad that the keyword tool is leaving HubSpot altogether.

I know I can not affect this decision, I don’t intend to try, but I do believe that there is value in a keyword research tool within HubSpot for its users - even in an era when topics, pillar pages and content clusters are the path to success.

Afterall, topics are simply head terms, or short-tail keywords. But what we still see time and time again is that businesses can fall into a trap of choosing the wrong topics. Internal language often differs from buyer language and egos result in the use of brand names over search terms, meaning a content strategy could easily miss the mark. As an agency, we often have to educate our client on this.

Keyword research that combines search volume, difficulty and an indication of current and competitor rank, combined with some lateral thought, can help users to select the right topics to go after.

I whole heartedly agree with the topics and content clusters approach to SEO. In fact we’ve be operating along those lines for some time. But keyword data helps us to ensure the success of that approach.

Of course, we can go and get data from other suppliers, but it was nice to use HubSpot as a one-stop-shop.

Oh, and I thought it was interesting that in that case study on IDS, there is a screenshot of their first page Google ranking as a measure of success :wink:

Here at Blend we absolutly love HubSpot; the product, the people, and the company. And this will not change that. But I wanted to share some thoughts with you.

Cheers, Phil.

3

Phil - this is an excellent point and a great place to jump into a discussion. As you correctly state "Keyword research that combines search volume, difficulty and an indication of current and competitor rank, combined with some lateral thought, can help users to select the right topics to go after” we agree.

The problem is, our own Keywords tool has had its limitations in tracking rank and difficulty for some time, so much so that a significant number of partners and users have elected to use their own keyword tracking tool of choice (SEMrush for example) on top of the Content Strategy framework. We see this in feedback and declining usage of the Keywords tool. Because we want to deliver the unique value of topical performance over time so marketers can understand the true impact of their content (since we built on top of our own CRM and reporting platform) we are 100% are in favor of turning to solutions specifically tailored to keyword research for that job.

Finally, we will definitely continue to explore and build new and innovative ways to connect topic discovery to content efficacy within the Content Strategy tool. That way, HubSpot can still be a one-stop-shop for planning, validating, executing and measuring your content marketing. In full transparency, this update gives us the focus to build something remarkable for the next 20 years of SEO, rather than the next couple. Thanks again for the discussion and feedback.

4

Hi @Jeffrey,

I share Phil’s sentiments about Hubspot choosing to discontinue the Keywords tool.

From the perspective of a direct consumer, rather than an agency, keyword ranking continues to play a decisive role in stearing our content campaigns in the right direction. Similar to Phil, we use keyword metrics, in large part, to ensure we’re using the right language in our content--both in terms of what our personas are searching for and ranking competition.

Keyword strategy goes very much “hand-in-hand” with our topic cluster strategy we’ve used. And to say topic clusters signal the end of keyword strategy doesn’t match my experience seeing both work well together in practice.

I’ve not seen issues with the accuracy of keyword reporting from Hubspot. And I’d remind users that Hubspot claims to get their data directly from SEMRush--it says so explicitly in the tool. So I’d reconsider switching to that service if you had doubts about Hubspot’s reporting accuracy.

One free tool I used before adopting Hubspot is SEO Centro keywork rank checker. I found it to be very similar in data when compared with Hubspot’s tool about 1.5 years ago.

5

I think that what you are saying is that this is a difficult issue and the Hubspot does not want to devote the resources to improving it. Because it is difficult does not change that it is important to many of us.

6

Honestly, I’ve been using the topic cluster since it was released, but the suggested topics always seem irrelevant to the core topic. Suggested topics don’t give Marketers any context on difficulty, monthly search volume, where competitors rank for these topics, etc. The fact that we will have to go to yet another costly tool (SEMRush) for keyword research is really disappointing. As someone alluded to earlier, keyword research is an essential part of building a topic cluster strategy. They do go “hand-in-hand.” The beauty of HubSpot is that it’s an all-in-one marketing tool.

I loved to use the keyword tool and felt like there was a great deal of value having it as a part of our subscription. Nobody can expect keyword ranking to be spot on every time. Search engines change all the time. Perhaps some of your newly onboarded customers have unrealistic expectations related to keywords.

But, it is helpful to see where you’re at and the long-term progression of your keyword growth. And, like I said before it was helpful for research, understanding difficulty, and seeing where competitors rank on various core terms. I’m incredibly disappointed that HubSpot is completely sunsetting this useful tool.

7

I’ve run into the same issues, thus the reason I’m booking an apt. w/Customer Success Team before keywords is removed to help adjust what may or may not work for us.

Hope this helps.

8

I agree with the contributor who suggested Hubspot buy us all a subscription to SEMRush- or incorporate the tool into your tools. Even if the Hubspot feels that the keyword tool is not 100%, we use it in our own way. If I see a trend of a keyword declining over time, I take action on the affected page. This will be a big loss for us. What is the harm in leaving it? I check our rank against the tool and the tool is right on most of the time.

Our website is information with a vast number of products. Most of our content is on the product pages not in blogs. We don’t write content based soley on keywords. Never the less the tool is very helpful.

Could you leave the tool for your current subscribers?

Thank you for responding to the post in this forum.

Best,

Nana Hinsley

9

Phil,

I completely agree with your points. I find that the KW tool in HubSpot, even if not the most accurate, provides illuminating color to my research and topic ideas. If the topic clusters are supposed to be doing that, I don’t feel they do it well yet.

I take issue with one of the screen shots as well. Adam Lasnik specifically states that he doesn’t, “speak for Google officially in this area anymore.” This indicates that he wasn’t Google’s rep for quite sometime seven years ago. This is not what the post indicates in the sentence that immediately follows, “This image is important because even 7 years ago Google was directly telling us we shouldn’t focus on rank.” (emphasis mine).

Google wasn’t telling us rank checking was a waste of time. An employee at Google was sharing his personal experience and thoughts.

I hope that HubSpot reconsiders their decision, or gives us more information on what they’ll be doing in the future. This is quite disappointing for a new customer.

10

Hi @seanpomory. Thanks for the comment and feedback.

I agree that keyword research can still be a valuable and illuminating part of the process. I broke it up into 4 categories in my full response below (here), and where I think this information is most helpful is validating the topics and keywords within a topic that you are going after.

Regarding the validity of rank tracking, you are correct that Google overall has not said that “rank tracking is dead”. That said, Adam Lasnik was a Search Evangelist from 2006-2010, and he posted the Quora response less than a year following, which I believe gives him a lot of insight into how search operated at the time and metrics marketers were paying attention to. In addition, there have been numerous SEOs, including Rand Fishkin from Moz debating obsessive rank tracking.

11

Hi Phil,

I wanted to offer a solution to you and anyone else that repsonded here that they were upset by this descion. Like you, we are an agency partner, and while we have been implementing content pillars, cluster topics etc for a while now, it is still important to us and our clients that we track and measure progress.

For that reason, we have created our own technology called Clickx.

We have been moving all of our clients KW data over to Clickx, in order to track rankings and additional SEO data at the page level.
To be clear - this is not a replacement for HubSpot, but it will replace (and improve) the way you are currently tracking keywords and on-site seo data.

As I mentioned, we are a hubspot partner - so we built Clickx with hubspot in mind, in an attempt to supplement what HubSpot had to offer.

We can offer these SEO tools, including the keyword tracker to agencies, as well as users. Please let me know if you are interested in learning more about features, pricing etc.

Thanks,

Taylor

12

I totally agree with what Phil has said and I’m very disappointed that Hubspot is retiring the Keyword tool. We do still use this as a starting point for determining what topics our content should be focused on. And, right or wrong, we still have “keyword research” as part of the services we provide to clients around blog development and removing this tool forces us, as an agency, to use other tools to fulfil this part of our SLA with customers.

13

Hi @Jeffrey - While I share the sentiments of others who find the keyword tool’s simplicity helpful, and am frustrated with the generalization that a keyword based approach is no longer relevant, I have a few specific things to bring up:

1. How will this impact the optimization information on pages and blog posts? I know right now that is linked with the keyword tool. Will the optimization feature still exist once keywords are removed from HubSpot?

2. The Content Strategy Topic Clusters (a tool I have been mostly happy with so far) often runs extremely slow. With this push to get more people to use it, I hope the devs are working to make that tool run better.

Looking forward to hearing your thoughts!

Bri

14

Jeffrey,

Like the others I can’t say I’m happy about this decision. It’s like a good friend enthusiastically announcing that he is dropping out of medical school to go live in a commune and grow organic pineapples or something. He’s very excited about the change. He provides all sorts of rationalizations for his decision, but you’re left thinking, “Hmm, I don’t think that’s really a good decision and I’m having a hard time telling him that I support him.”

Specifically, I never really noticed any issues with the keyword information that Hubspot was providing. It seemed to be pretty accurate to me and gave me an understanding of how some of our content was performing. Occasionally I would run keyword searches to verify the information provided and it seemed to match what was being reported. For instance one of our e-guides changed our search results for a particular keyword phrase from about 40 to #3. I could do the search myself and verify that information. To me that sort of data was very valuable.

OTOH, if I go to my topic clusters I see that topic cluster A has increased search volume of 22% versus last month, but I don’t know how you reached this conclusion. Had those pages all existed and not been connected together by hyperlinks would they (as a whole) have had the same traffic? Is the mere act of linking them together in a cohesive format the sole reason for this reported increase in performance or did one of the pages simply creep up in search results on its own and up the volume for the topic cluster as a whole?

I simply don’t find this data to be as useful and actionable as the keyword data.

So, Jeffrey, quit medical school if you must, but I don’t support your decision. Sorry.

15

Bottom line is we PLAN content strategy by topic clusters but we MEASURE it with keywords. I feel Hubspot is using the wrong narrative to justify a rather questionable change.

16

Hi All,

Thank you for the feedback. I want to take a few minutes and try to address all of your points.

Before that though, I want to be clear at HubSpot we are not saying that keywords are no longer relevant, in fact, a topic is generally made up of hundreds or even thousands of individual long-tail keywords. Indeed, part of the difficulty with this transition has been keeping keywords as a function to a higher standard, while deprecating the Keywords tool.

That said, @jc summarized it well that keywords are primarily used to plan what to write and then measure the success of content. In order to cover all of your feedback and comments we’ll break this down slightly further into planning what to write, validating what to write about, executing that you are on the right track, and finally measuring how you are doing. We’ll dive into each of these in more detail below.

Planning

The product updates blog post, and numerous other research and resources across the web have talked about how a keyword-based approach is no longer as effective as it once was. One of those reasons can be seen within our own customer base. Many Keywords tool users end up spending hours of time first finding the right list of keywords (from outside of HubSpot to begin with), gathering about 300 or so that may or may not be relevant to their business, and attempting to writing blog posts for each. That takes a lot of time and energy and scatters content across numerous topics that don’t provide a clear signal to Google (or your prospective customers) what your business is an authority on or should be known for. With the move to focus on topics first, we can start with a topic a business wants to be known for in a planned and straightforward manner that is not spread across numerous keywords. By starting to answer a question, “what does your business want to be known for?” the answer to that question is likely a core topic, and Content Strategy can help you holistically plan around that topic, and suggest additional topics that relate to longer-tail keywords.

Validating

Next, you need to validate the terms you’ve chosen will be competitive, relevant, and popular enough to spend time creating content on. Admittedly, this is where the Keywords tools shines at first. For example, you could log into Keywords and get information on difficulty, search volume, and more in a simple format as many here have mentioned. That’s hard to find across other Keyword tool vendors largely because it’s simplifying a lot out of the equation for search in 2018. The Keywords tool has unfortunately not been able to reliably get non-SEMRush data for some time, the data that comprises difficulty and rank.

As we continue to work on being the one-stop-shop in Content Strategy (which currently gives you Monthly Search Volume, Relevancy, and Domain authority) we recommend using a tool like SEMrush to conduct keyword research validation within an overall topic should you find more information necessary.

Executing

Once you have planned and validated content, it’s time to actually start writing. This is where the SEO optimization panel that integrated with Keywords comes-in. Within this panel, we would ask for Keywords and then recommend using it numerous times through the page, and other on-page SEO advice -- much of which is outdated.

If we wind the clock back 8-10 years ago it would be common for marketers to choose a keyword and then sprinkle the exact same term through a post to signal to Google the page was relevant for that term. Today though, Google no longer operates like that or evaluates results using the same criteria.

We are in the process of rebuilding the SEO optimization panel now and will release it to all customers as soon as possible. This new version will refocus on incorporating topics, and ensuring that content fits into your overall topic cluster. This, combined with SEO recommendations in the actual topic cluster editor, allows for clearer “next steps” then the Keyword tool currently does; add in the execution play of creating pillar pages/internal links and that’s where we start to see customers displaying measurable increases in quality and quantity of traffic.

Measuring

We all know that since late 2013 Google has encrypted search results, which means that we no longer get specific data on which keywords people are searching for, and clicking on to land on our site. So, the Keywords tool and even Sources report diverged from measuring exactly what content brought what leads, and ultimately customers.

In place of this, many people have turned to rank as a measure of success. And yes, we have used it as well at HubSpot.

But the fact is, with the localization and personalization of search today rank is simply not a metric that we should solely define success on. One study showed that there is 0% consistency in rank. In addition, there have been numerous articles around the problems with rank tracking like this one, and even a Whiteboard Friday from Rand Fishkin.

While this is hardly a black and white issue, we do strongly believe that measuring the success your content by what topics generated sessions, leads, and ultimately customers is a more future-proof approach. This is possible today with Content Strategy, and we are investing more into in the future as well to clearly display the efficacy of the content generated across a marketing teams efforts - all efforts - not just blog posts and website pages!

Answers to a few other questions…

What about the SEO optimization panel within Website Pages, Landing Pages, and Blog?

We’re in the process of refreshing this panel now and plan to release a refreshed version shortly. The new version will focus more on topics and linking your content into topic clusters within Content Strategy, while also recommending on-page best practices.

I see that topic cluster A has increased search volume of 22% versus last month, how did you reach this conclusion?

There are two parts to this. The act of internally linking numerous pages/posts back to a central topic-based pillar page can in-fact increase your visibility in search results. This ties back directly to the execution step discussed earlier. When you look at two case studies from IDS Agency, and Townsend Security, these are both customers that have been creating content for a considerate amount of time now and when they adopted this approach have seen a remarkable increase in traffic.

The second part is that we are tracking traffic to any post, or page, within the topic cluster. The actual report that displays this change in sessions is the Topic Clusters report found in Reports > Analytics tools > Traffic Analytics.

Here are also some additional resources around this:

I hope this helps. I and the team here at HubSpot know this transition isn’t easy so my hope is that the context here describes how we’re thinking about SEO and supporting all customers, partners, and ourselves for the next generation of content creation.

If you have any other questions let me know. The product team and many others are listening and appreciate the feedback. Thank you.

17

Like so many others, I get the rationale posited, but am frustrated with the decision (and we appreciate the Strategy tool.) Multiple times you’ve recommended that we invest in a tool like SEMrush. We do use other tools in addition to the H.S. Keyword tool, but if your response to “Why are you taking this away?” is “Just spend another $100-400/mo to replace it,” is HubSpot prepared to reimburse loyal H.S. customers for that amount (rhetorical question)? As an agency, the keyword report is just one of the tools our CUSTOMER use to gauge how well we are performing on their behalf, e.g. we have a client who had virtually no keywords on page one when we started, who now has over 100, and they rarely have access to other tools outside of HubSpot. I even wrote a blog years ago on how much I love little green triangles, because it was such great visual feedback on progress or lack thereof, even though it understandably jumps around. My bottom line question is… WHY? We love to use HubSpot’s tools to sell HubSpot in part by showing them how easy it is to understand and use (and because it is all-inclusive). Please don’t make it more difficult for us to do so by removing features we use to help sell your products.

18

Yes!

19

Hi @jdthomas. Appreciate the comment and feedback.

One thing that I think is helpful is to take a step back and look at how content and SEO has evolved. For the sake of this, let’s jump back to 2006. At the time if you were creating content online you could sprinkle a single exact match keyword throughout a page and have a very good chance of ending up on the first page. Fast forward to the “golden age of content” in 2011 we could all clearly see which specific keywords were being searched for, and which of those keywords drove visitors to our website. But all of that came crashing down in late 2013 when Google started to encrypt search results and we all lost access to that Keyword data. At the time it was widely proclaimed as “the death of SEO”. At roughly the same time, it seemed like Google went to war against SEO, and more specifically, rank tracking.

Jon Henshaw, the founder of one of the first rank tracking solutions, Raven Tools, wrote an excellent post on comparing scraped rank data versus average rank data. Here are a few notable excerpts:

It’s important to point out that rank monitoring is not essential for doing effective SEO. It may be essential for some advanced analysis and tactics using formulas on a spreadsheet, but it’s not necessary for most activities. An SEO needs to optimize a site for search engines, find relevant and authoritative sites and build relationships. Sprinkle in a little content creation and social sharing, and you sum up – at least at a high-level – what modern SEO looks like.

Jon continued on to talk about measurement and relying on core metrics like traffic, conversions, and customers.

Ranking well for certain terms doesn’t tell SEOs they’re doing a good job. Organic traffic increases and conversions from that traffic tell SEOs they’re doing a good job.

Lastly, Jon mentioned that this is indicative of a much larger shift that is happening.

“Based on the information we had from reliable sources, we determined that this was the beginning of a much bigger (albeit very slow-going) fight against companies using scraped data. We didn’t see it as an AdWords issue; we saw it as a Google issue.”

This post is from August of 2013, but the quotes above (and most of the other information in the post) still hold true five years later. We’ve seen additional changes beyond just rank through the Hummingbird and RankBrain updates. While neither has caused the same immediate near-term pain that the loss of Keyword data did, it’s an undeniable evolution in how Google considers SEO and content should be done. The same way that marketers created content in 2006, 2011, or 2013, would not be nearly as effective today largely due to these two algorithm updates.

We really believe that Content Strategy is aligned with the way search works today, how searchers look for content (through pillar content, and better information architecture), and offers better insight into success through the Traffic Analytics reporting which gives you data on the number of sessions, leads, and customers a topic cluster, and each piece of content, is generating. Because this data is available, the intention is to make it easier for an agency (and any marketer directly working at a company) to show the results of their efforts.

While all the above talks about what has happened strategically, I also understand there is a tactical component and many today still look towards rank as a measure of success. Hopefully the post above from Jon Henshaw, and this one from Matthew Barby, can help describe why we all need to move away from this metric. Convincing your manager, or client to do this will likely take time and requires focusing on on the metrics that truly are a measure of success - links, traffic, and conversions.

I’m happy to talk further and share more information on how agencies or direct marketers can use Content Strategy and transition from Keywords. For agencies, I certainly understand there is added complexity of demonstrating these tools to clients, feel free to reach out to me via DM and I can send over a number of resources and hop on a call to walkthrough this. Here are a few resources in the meantime that can help with the transition and ensuring that you are creating effective content.

20

Jeffery,

I am in an engineering environment. The keywords are fairly well defined. It would be great to be able to build clusters around the content we created in the COS, but the tool only suggests Blogs and does not allow us to navigate to the COS and files create cluster links.