HubSpot Ideas

seobrien

More efficient way to find and merge duplicate contacts

The Sales CRM functions more as intended for lead capture from a web form where one email would be provided to create the contact record.  

You're thinking in terms of the CRM as a Sales and Marketing tool, which it is, but it's also just a CRM: Connected to Gmail it's getting new records daily from people that have multiple email addresses.  Moreover, just doing an import of addresses from other databases can result in multiple Contacts being created for the various email addresses someone might have.

Consider myself as an example, I have 12 different email addresses.  When I take a new client or job, I'll have another.  Those are 13 different records. They are all just me and each of those email addresses should be more easily uncovered as associated with the same person and merged.

 

What needs to improve
Today we have to go into a Contact records and use the Merge function to find the other emails and merge them.  That's a manual process. A pain for large Contact databases.

The Contact list should have a "Find and Dedup" option.  The CRM should find likely duplicates not based on the email identifier but other personal identifiers: same name, similar name in same location, etc.  Flag those in the list and make it easy to check the duplicate Contacts and "Merge" them.

This really needs to be done as part of the platform as growing companies, teams with many people using Gmail as part of their outreach, etc. will constantly result in new Contact records that can go unnoticed as duplicates of existing records.

HubSpot Updates
July 30, 2019 06:02 PM

Hi Daniel -- I've reached out to schedule a call for us. Thanks again

July 30, 2019 05:26 PM

Hi folks -- thank you for all of the continued feedback on the duplicate management tool! We're thrilled that this has been as useful as we'd hoped to help businesses keep their data as clean to take advantage of a unified customer database for marketing, sales, and customer service teams.

 

We're proud to build an open platform to support our integration partners like Dedupely and Insycle. While our partner products can deliver differentiated value, the native duplicate management tool is developed entirely by HubSpot.

 

How we identify potential duplicates: Under the hood, this tool uses machine learning (ML) to identify contacts and companies that you are likely to merge. At its core, ML helps automate tasks by analyzing examples of a task to evalutate new tasks to complete. In the case of duplicate management, the task is merging contacts. By analysing past merges, ML algorithms identify other pairs of contacts that have a high likelihood of being merged based on your behavior, which are shown to you in the duplicate management tool.

Today, the ML models for contact and company merge suggestions are based on contact and company properties including name, email, phone number, company name, company industry (determined by HubSpot Insights), and company industry (determined by HubSpot Insights). We plan to expand the properties that the ML models consider to other properties, but to be transparent, we're more likely to add default contact properties (e.g. contact activity data) before custom portal properties in the near term.

 

Because this is a machine learning product, it doesn't rely on strict rules like exact matches phone number or name; by not relying on rules, the duplicate suggestions can more closely mimic our own human understanding of challenges like several contacts with the same business phone number or several contacts with the same generic name, like Kevin Smiley Happy.  As you merge and dismiss pairs, you provide feedback to the tool to help improve the accuracy of our merge recommendations based on patterns in the contact and company properties.

 

This tool is still under active development, and you should expect improvements around both the accuracy of the suggested merges and the merge experience throughout the rest of the year. 

July 12, 2019 02:33 PM

Thank you! We're proud to be building an all-on-one platform to allow businesses to combine HubSpot's suite of tools with their choice of powerful integrations, like Dedupely. It takes a village to keep CRM data clean to enable millons of businesses to grow better. 

- Kevin Walsh (senior product manager @ HubSpot)

June 13, 2019 03:29 PM

Thank you so much @ck2018 ! We appreciate the kind words, and we appreciate being able to work with great, value adding partners like Insycle. We're eager to continue building a strong, loveable platform for engineering teams like Insycle to help us help millions of buisnesses grow better.

 

- Kevin Walsh (senior product manager @ HubSpot)

Delivered
June 12, 2019 04:49 AM

Hi HubSpot Community,


Thank you all for the continued feedback on this post. The continued insights into tools needed to help your businesses grow better is exactly what we need to build great products. 

Today, I'm excited to let you know that you’ve got a brand new tool that finds duplicate contact and company data in HubSpot. No extra spreadsheets, tools, or costs. So you’ll be more efficient, and your customers will have more frictionless experiences with your brand. This tool is now available for all professional and enterprise customers - full details on how to use it can be found in this knowledge article


This product leverages machine learning to consider data such as name, email(s), IP-derived country, phone number, zip code, and company name when comparing two objects. When you accept (merge) or reject (dismiss) a pair as duplicates, you’re providing feedback to the model to help it improve over time. We're likely to add more data to the model in the future.

 
Again, thank you all for your continued feedback on this idea. Your use-cases, examples, and urgency help us build better products. Happy deduping! 

April 14, 2019 06:32 AM

Hi all -- We're excited that we're getting closer to being able to release the new tool to all customers. We're working to scale its processing, so that it works for everyone, including customers with large numbers of contacts. 

 

The link in BB1's post will only work if your portal has been accepted into the beta program. If you are interested in becoming an early user - please fill out the beta form here. We will be in touch if you are a good fit.

 

We won't be accepting every submission into the beta, but we will reach out to submissions that are a good fit for the early version of the tool.

 

Thank you

January 08, 2019 04:47 AM

Hello HubSpot community - I wanted to re-illustrate that this tool is available from HubSpot in private beta. Currently, the beta supports contact duplicate identification. We hope to introduce companies duplicate identification in the coming weeks as well. 

 

If you are interested in becoming an early user - please fill out the beta form here. We will be in touch if you are a good fit.

 

Note: We will not be accepting every submission into the beta, we will reach out to submissions that are a good fit for the early version of the tool. 

Re: More efficient way to find and merge duplicate contacts - changed to: In Beta
November 20, 2018 11:35 AM

Hello HubSpot community - I'm excited to let you know we have an early version of this tool available in a private beta. If you are interested in becoming an early user of this product - please fill out the beta form here and we will be in touch if you are a good fit.

 

Note: We will not be accepting every submission into the beta,  but we will be reaching out to submissions that are a good fit for the early version of the tool.  

October 29, 2018 12:08 PM

If you are interested in becoming an early beta tester of this product once it is developed - please fill out the beta form here.

Re: More efficient way to find and merge duplicate contacts - changed to: In Planning
October 29, 2018 11:45 AM

Hi HubSpot community, 

This is something the Product and Engineering team is beginning to research and plan, we hope to deliver a solution in the coming months. 

Investigating
April 03, 2017 10:53 AM

Thanks for adding this idea. Helping customers identify and merge duplicate contacts and companeis is somethign that we are putting a lot of thought into. We appreciate the feedback and examples, this is extremely useful.

143 Replies
809CharlieTango
Participant

Does anyone know if there is something similar to RingLead, but integrates with HubSpot? https://www.ringlead.com/marketo/#.WbARDNOGNE4

Fitzwalsh
Member

Can't wait for this new feature!

DanOasisFun
Member

To lower possible programming and user frustrations, maybe not relying on the import to catch duplicates, but a button to select to search for duplicates and some how see some details of the contacts so we can verify if they truly are duplicates or if actual different contacts, or to see and select to merge the information of the contacts. 

 

We had it where one associate imported contacts and another associate imported contacts, as we got started with the program, and there are duplicates.  Some with more information than others.

tabbs
Member

Is there any movement on this feature?

As a company who's fees are largely formulated based on the number of contacts in our systems, this is a critical feature in which you should already be providing to us. It is also one of the most basic features found in almost any CRM in which the purpose is for correctly tracking contact and interaction history. 

This is a very horrible inefficienty, but that it also means for us additional sunk costs in the time it takes to manage manually as well as how your monthly fees are structured is just salt in an open wound.

realizeit
Top Contributor

This is a must have--especially for those of us paying for the CRM. We need an easier function to find duplicate names and merge them, either by filters or some other EASY process (and not within the Marketing side of things, either). Who the heck needs Snippets when this duplicate pairing function is a core of most good CRMs? Do you folks at HS look at the number of requests for features before placing new features in the "upcoming" queue? Do "needs" surveys?

jameslyons
Participant

HubSpot, it's been many months and a lot of people have voiced their support of this change. Any updates at all?

malcolmslater
Member

I developed a dedupe feature for data which I roadtested in several lrage public sector organisations over the last few years. Just wondering if there's enough support here for it to be worthwhile creating an extension to hubspot?

DanAt3DV
Contributor

@malcomslater - this would certainly be beneficial. Happy to beta test...

jameslyons
Participant

@malcomslater - we would very much appreciate the extension you described. Thank you!

realizeit
Top Contributor

As long as it's simple to use w/step-by-step instructions, would be great!

Leahm14
Member

I have to voice the same concern many others have voiced. Since you charge users based on contact volume, providing a deduping option seems like it would be a logical (and morally sound) function. 

 

Dirty data is an issue database management teams have had for years, and Hubspot's lack of dedup functionality seems really behind the times.

809CharlieTango
Participant

 @malcolmslater you bet it's a big requested feature. To have, doing it manually once a month sucks. Takes up about 4 hours to go through a list in excel, setting up formulas, etc. Even setting up formulas isn't a foolproof way of doing it because then I have to go back and check if a duplicate record is truly duplicate or if a contact just happens to be someone with the exact same name and an entirely different email address. 

 

Our company works with school districts and higher education and our contacts can be the same or different depending on the email address our contacts use. I would be cool if a feature would scan through the list based on the region, location and/or IP address of the contact to notify a HubSpot user to merge or check and see if a contact is duplicate or not. 

 

Thanks, 

Orlando 

jakekirsch
Participant

Strongly agree with everyone voicing the need for deduping in hubspot using more criteria than email.

 

Case scenario.

Contact requests info from our company and providers email 1. (personal email for example)

Later contact purchases from our company and we receive a different email for the same contact (work email for example)

 

Our company continues marketing to this person via Email 1 and We look like a dumbass.

 

Our database is around 100k contacts. Even exporting to excel and updating customer contacts with additional email addresses, then deleting duplicate contacts is a task BEYOND painful and time consuming.

 

How is this not a HUGE need for every company?

 

How has @Hubspot ignored this issue?

 

Could it be the basis of hubspots billing method? More contacts = More money.  Positively!

 

 

 

 

mattmayo2
Member

Please implement this.

martin_montreal
Member

yes! how does one vote Yes, yes, yes asap!!! 

Sfahl
Member

I'm a new user and am very impressed. Well, at least I was. The inability of CRM to display duplicates or possible duplicates is amazing to me. Especially since your users have been requesting it in a big way. 

 

In my scenario: One of our current customers signed up for a webinar with a different email address. She is now part of multiple work flows for non-users complete with sales tasks (such as calling).

Negatives:

1. It's hard enough to get sales persons to use a CRM, add to that they are now calling people who are already clients and the problem multiplies.

2. Very annoying for a current customer to get marekted to as if they aren't a customer. We lose some or all the hard work we put in to make our customers feel specail.

 

 

RiccardoPisani
Top Contributor

+1

jmihaljevic
Member

Just found this thread as duplicates are a major point in my database of 45,000 records.  Please hurry up this essential feature!

kgrider
Participant

Yes, this would be great! We have so many duplicate contacts in Hubspot because they have multiple email addresses. 

Coco2302
Participant

shocking to see how you guys just ignore this thread. Quite disappointing for a company that publishes books on how helping the customer and providing great service is are ultimate objectives...