Salesforce Duplicate Records: Find and Fix Before AI Scales the Damage
Your CRM says 12,000 contacts. You might have 8,000 people.
Part 2 of the Clean Data for AI series | By Jeremy Carmona
Your CRM says you have 12,000 contacts. You might have 8,000 people.
I saw this on a migration engagement. A client moved 70,000 records from a legacy system without running a deduplication pass. The result: 12,000 duplicate Accounts, 8,000 orphaned Contacts, and 3,200 Opportunities with amounts stored in the wrong unit because the source system used cents and Salesforce uses dollars. The annual revenue dashboard showed $4.2M in pipeline. The actual pipeline was $42,000. The board had been reviewing that number for two quarters. Cleanup took 6 weeks, 2 consultants, and $25,000. The deduplication that would have prevented it would have taken 4 to 8 hours.
That gap between what your CRM reports and what is real is your duplicate exposure. Every record in that gap inflates your pipeline, distorts your reports, and confuses your routing logic. When a human reviews the report, they might catch the problem. When an AI agent acts on the data at machine speed, the problem compounds before anyone notices.
Why Duplicates Hit Harder with AI
Duplicates have always been a data quality issue. What changed is the speed and confidence of the systems acting on the data.
An Agentforce agent that routes a case to the wrong contact does not pause to check if two records are the same person. An Einstein recommendation built on inflated pipeline does not flag that 30% of the Opportunities are duplicates. A Data Cloud segment that counts two records for the same donor reports the wrong headcount, and the board makes decisions based on a number that was never real.
Before AI, a duplicate was an annoyance. A rep would notice two records and merge them, or not, and the consequences were local. With Agentforce, Einstein, and Data Cloud operating at scale, every duplicate is a silent multiplier. The agent acts on both records. The model trains on both records. The report counts both records. The damage is no longer local. It compounds.
Across 30 assessments, duplicate ratios on Contact records sit between 8% and 22%. My AI Readiness Checklist sets the red flag at more than 15% of records being duplicates. The acceptable target is below 5%. Most organizations have never measured their number.
The Excel Diagnostic
You can measure your duplicate exposure in under an hour. You need a data export and Excel.
Step 1: Export and backup. Export Contact records including: FirstName, LastName, Email, Phone, AccountId, CreatedDate, LastModifiedDate. Save a backup copy before any analysis.
Step 2: Quick duplicate scan with Remove Duplicates.
In Excel, select the Email column. Go to Data > Remove Duplicates. Select only the Email column. Excel will report how many duplicates it found. Do not save this change to your working file. This is a count, not a cleanup.
Record the number: original count minus unique count equals your email-based duplicate exposure. If you started with 12,000 contacts and Excel reports 9,600 unique emails, you have roughly 2,400 potential duplicates, a 20% rate.
Step 3: Cross-field matching with a composite key.
Email matching misses duplicates with different email addresses. The same person with a work email and a personal email shows as two records. To catch those, add a helper column that concatenates first name, last name, and phone:
=LOWER(TRIM(A2))&"|"&LOWER(TRIM(B2))&"|"&SUBSTITUTE(SUBSTITUTE(D2,"-","")," ","")
LOWER normalizes case. TRIM strips leading and trailing spaces. SUBSTITUTE removes dashes and spaces from phone numbers so that (555) 123-4567 and 5551234567 match as the same number.
Run Remove Duplicates on this concatenated column. The additional duplicates found are records that share a name and phone but have different emails. These are the duplicates your email-only match missed.
Step 4: Assess the impact.
Create a Pivot Table grouping duplicates by Account. The Accounts with the most duplicate contacts are your highest-risk areas. If those Accounts have open Opportunities, the pipeline associated with them may be inflated. Sort by count, descending. The top 10 Accounts on this list are where your AI outputs are most likely to be wrong.
Common mistake: Running this analysis on only one object. Contact duplicates are visible. Account duplicates are expensive. Lead-to-Contact duplicates are the ones most organizations miss entirely. Run the analysis on all three.
What Salesforce Should Prevent Going Forward
The audit finds the current state. Prevention handles the future. Salesforce provides two native tools for duplicate prevention: Matching Rules and Duplicate Rules.
Matching Rules define the criteria for identifying potential duplicates. The default Standard matching rules use fuzzy matching on name and address fields. Most organizations never customize them.
To review: Setup > Matching Rules. Check which rules are active. Check whether they cover the fields that matter for your org. If your primary identifier is email (common in B2B), you likely need a custom matching rule that includes Email as a matching field. The default rules do not cover email.
Duplicate Rules define what happens when a match is found. The options: alert the user, block the creation, or log the duplicate. Most organizations set them to "alert" and wonder why duplicates keep entering the system.
The governance decision here is straightforward but most teams avoid it: should duplicate rules block creation or alert? Blocking prevents duplicates but may frustrate users who encounter false positives. Alerting allows creation but relies on users to check. The right answer depends on your false-positive rate. If your matching rules are well-tuned and false positives are rare, block. If you are still refining the matching criteria, alert and review.
The Review Process Most Organizations Skip
Salesforce creates Duplicate Record Set records when duplicates are detected. These records exist in your org right now. Most organizations have never opened them.
Go to Setup > Duplicate Record Sets. Sort by creation date. The volume tells you how active the problem is. The age of the oldest unreviewed record tells you how long it has gone unaddressed.
One organization I worked with had never opened the Duplicate Record Sets object in Setup. When we looked, there were thousands of flagged records going back years. No review process, no merge decisions, no owner. The detection was working. The response was not.
A data steward should review Duplicate Record Sets weekly. The review produces one of three decisions: merge (the records are the same person), keep separate (false positive), or investigate (not enough information to decide). Without this review cycle, detection accomplishes nothing. You are collecting evidence of a problem and filing it where nobody looks.
After implementing active matching rules that blocked duplicates at entry, a weekly review cadence for flagged records, and a named data steward who owned the merge-or-keep decision, that organization's duplicate creation rate dropped to near zero on new records. Reporting accuracy improved in the first month because the pipeline was no longer inflated by ghost records.
Risks and Limits
Merging records is destructive. Once merged, the secondary record's data is partially preserved in the surviving record, but some field values are lost. Before any merge operation:
Back up the data
Test merges on a sample of 10 to 20 records first
Preserve original values in a notes field or related record
Review Opportunity and Activity relationships before merging Account records
Fuzzy matching is imperfect. It will miss some duplicates and flag some false positives. The goal is not zero duplicates. The goal is a known, measured, managed duplicate rate with a named owner who reviews the flagged records on a schedule.
What to Do This Week
Export your Contact records with name, email, phone, and AccountId
Run the email-based duplicate count
Run the concatenated name-plus-phone duplicate count
Pivot by Account to find highest-risk areas
Review active Matching Rules in Setup
Check Duplicate Rule settings (block vs. alert)
Open Duplicate Record Sets and count unreviewed records
Assign a weekly review owner
The duplicate rate you find will tell you how much of your pipeline and reporting is real, and how much is noise the AI will scale.
Related CCC resources:
Salesforce Duplicate Management Guide (comprehensive admin reference)
AI Readiness Scorecard (free, 2-minute self-assessment)
Nonprofit Data Quality Audit Template (NPSP-specific scoring spreadsheet)
Sources:
Salesforce Help: Duplicate Management documentation (help.salesforce.com)
Gartner Press Release: "Through 2026, 60% of organizations that adopt AI will be forced to roll back or abandon projects due to data quality" (February 2025)
CCC assessment methodology (30+ field assessments, 2024-2026)
This article is Part 2 of the Clean Data for AI series. Part 1 covered measuring AI-ready data. Part 3 covers placeholder values that pass validation with zero information.
Last updated: September 23, 2026. If you find an error, contact contact@clearconciseconsulting.com.
