In order to de-dupe successfully, you should first identify your business rules for establishing duplicates and for identifying surviving records. The second step is to review the data to make sure you have done so accurately.

Duplicates can be identified using different scenarios, one of which is using company name, website as well as address. In order to maximize your chances of find a duplicate, you should standardize/normalize those fields. 

For example, the same country may be represented differently in your data due to the input: Canada, CA, CAN, by normalizing them all as Canada you are now improved your chances. Normalization of a website includes removing http, https, www. and anything behind the actual domain name, this is probably the most important step in normalization. Company name normalization involves standardizing the company entity type, for example Corp. and Corporation would be the same thing. You should also normalize any other fields that will help you identify duplicates and the surviving record. StrategicDB’s Deduplication tool allows you to normalize: website, address and company name. 

So what are the steps to take to start you de-duplication:

  1. Pull all data from your CRM that you will require to: identify duplicates, establish the surviver and that will help you review your file. For example, when deduping account data you may wish to pull in account owner for secondary review, website, company name, address, phone number to establish a duplicate and # of opportunities for establishing if it should the master record that survives your deduplication process.
  2. Come up with your own company rules to identify duplicates in the de-duping tool. The rules can help identify your duplicates, they can be based on website, address, company name and as well as rules such as if the company name and country is the same than its a duplicate. If the same company in your file has different names but shows the same address you can set up a rule that a duplicate is only a duplicate if the address is the same, etc… The de-duping tool will allow you to combine multiple fields to increase de-duping accuracy. You can make your deduplication criteria as strict or as loose as you wish. 
  3. Identify rules by establishing a “Master” record: Your rules for establishing a surviving record among duplicate records can be based on record type, date created, how many opportunities or activities the record had, data completeness or just randomly if your duplication scenario is very strict.
  4. Review with the teamPrior to applying any live system changes always double check the list of rules you set up in the de-duping tool with your team to avoid errors prior to applying any live system changes. If de-duping will impact any specific team, it is advisable for that team to double-check that you do in fact  want to de-dupe those specific records. 
  5. Finally when all is double check, it is now time to implement those changes in your CRM or Marketing Automation Tool.

If you are looking for data deduplication tool to use your own business rules, check out StrategicDB’s de-duping tool here.