If CRM duplicates keep coming back, investigate the newest duplicates before merging another thousand old records. Find which form, import, integration, or employee created each pair. Then inspect what that entry point uses to decide whether a record already exists.
A merge tool can repair a backlog. It cannot compensate for an integration that creates a fresh company every time a webhook repeats, or a weekly spreadsheet import that has lost its identifiers. Those defects need different fixes even though the CRM screen looks the same.
The practical goal is one correct record for the entity your business actually tracks, with its history intact. A smaller record count alone does not demonstrate cleaner data.
Why another cleanup weekend may not solve it
In an r/CRM thread about merging duplicate accounts, u/neilsarkr described spending a Saturday merging roughly 400 accounts. Their frustration was the tradeoff between rules that miss obvious pairs and rules that produce too many questionable matches.
The replies included tool recommendations, several from vendors or consultants. One commenter said they did not encounter the problem and asked how the data was being ingested. Another, u/kate_in_tech, asked where recurring duplicates were coming from. That is the most useful investigative question in the thread. The reported workload is an anecdote, and the product recommendations are not an independent comparison.
The distinction matters when hiring help. A proposal to merge existing records should explain what happens when the next import or integration run arrives. Otherwise, the project may successfully clear a queue that immediately starts growing again.
First identify what has been duplicated
Two similar company names do not necessarily describe the same business record. Your CRM might deliberately track a parent company, a subsidiary, and individual service locations. Combining them can erase the structure sales and delivery teams need.
Similarly, two activities on one contact may be duplicate deliveries of the same call event. That is not a contact-matching problem. Merging contacts will not fix it.
| What you see | What to investigate | Evidence to preserve |
|---|---|---|
| Two contacts for one person | Identity rules and changed email addresses | Source IDs, addresses, creation channel |
| Two companies with one domain | Import or API creation behaviour | Original payload and integration owner |
| The same call logged twice | Repeated events or two logging systems | Call ID, event IDs, timestamps |
| A task recreated after deletion | Sync direction and deletion handling | Task mapping and sync history |
| Separate branches merged together | Wrong definition of an account | Location and parent-child relationships |
Write the entity definition before choosing a matching rule. “One account per legal company” and “one account per operating location” produce different correct results. A technically consistent rule can still be wrong for the business.
Trace a small sample back to its source
Choose recent pairs from different channels. Avoid beginning with the oldest, messiest records, whose history may no longer be available.
For each pair, collect the creation timestamp, creator or integration, original source identifier, relevant matching fields, and the first associated activity. Compare what the two requests actually contained. A spreadsheet might display a full email address while the imported column was blank or mapped to a different field.
Build a short origin log:
- Which event requested creation?
- Which system sent it?
- Which identifier did it send?
- What matching check ran before the write?
- What did the destination return?
- Did another process handle the same event?
If the logs cannot answer those questions, add traceability to the integration before guessing at a fix. Record identifiers and outcomes without copying unnecessary personal information into broadly accessible logs.
The sample is diagnostic, not a prevalence estimate. Finding eight duplicates from one integration does not establish that it caused every duplicate in the database. It does give you a concrete path to reproduce and repair.
Check the creation channel, especially in HubSpot
A platform's automatic deduplication is not necessarily identical across manual entry, forms, imports, and API requests.
HubSpot's documentation describes contact deduplication by email and company deduplication by domain in supported creation paths. It also states that companies created through the API are not deduplicated by the Company domain name property, including those created through installed third-party sync apps. Imports can instead use Record IDs or custom properties requiring unique values. HubSpot deduplication documentation
That exception explains a plausible failure: an admin tests an import, sees duplicates prevented, and assumes an API integration has the same protection. The integration may need its own lookup, stored identifier, or supported upsert operation.
Do not translate this into a blanket claim that HubSpot cannot manage duplicates. Inspect the actual object, creation path, and configured integration. Other CRMs also need their own documentation checked rather than inheriting assumptions from a different product.
Give established records a stable mapping
Once you know which source record belongs to which CRM record, store that relationship. Repeatedly rediscovering the same customer from a name is unnecessary uncertainty.
A mapping might connect a billing-system customer ID to a CRM company ID. Include the source system and account or tenant in the key so identical numeric IDs from different systems do not collide. Preserve the mapping when a display name or email changes.
For a new record, define an ordered matching policy. A trusted external identifier can be decisive. An exact email may be useful for an individual contact. A similar name and shared office phone might only justify a review candidate.
Do not turn every unmatched record into an automatic creation. “No match” can mean the source omitted a required identifier, an API lookup failed, or the existing record is inaccessible to the integration. Those conditions deserve different outcomes.
A good policy includes three explicit decisions: update a known record, create a genuinely new record, or stop for review. That third branch prevents uncertainty from being disguised as growth in the database.
Make repeated delivery produce one intended result
Integrations need to cope with events arriving more than once. Stripe, for example, documents duplicate webhook deliveries and recommends tracking processed event IDs; it also notes that events may arrive out of order. These are documented properties of Stripe's system, not proof that every connector behaves identically. Stripe webhook guidance
The broader engineering requirement is idempotency: repeating the same operation should not repeat its business effect. For a completed call, the result might be one CRM activity associated with the correct contact.
Store an operation identifier and distinguish work that is pending, completed, or awaiting reconciliation. Merely marking an event “seen” before the CRM write can lose work if the process crashes. Marking it only after the write can permit duplication if the write succeeds but its response disappears.
Your implementation needs a way to resolve that uncertain interval. Depending on destination capabilities, that may involve a supported idempotency key, a unique external ID, an upsert, or checking the destination before retrying. The developer should be able to explain the crash case, not just the happy path.
Test simultaneous requests and competing writers
A lookup followed by creation can still race. Two workers may both search, both find nothing, and both create a record before either sees the other's result.
Where supported, enforce uniqueness in the destination or use an atomic operation, meaning the check and write cannot be interrupted by another competing write. If the destination cannot provide that protection, serialize work for the same business identifier and reconcile uncertain outcomes. Ask for the limits of the approach in writing.
Also check whether two legitimate integrations both believe they own the same activity. A phone platform and a separate automation may each log completed calls correctly according to their own configuration. Together, they produce two activities.
Assign one writer for each record type or define how writers coordinate. This often resolves more confusion than adding another fuzzy matching rule.
Bring a recurring duplicate and the systems that created it. We can map the integration, matching rules, and exception handling around the actual failure.
Tell the integration when a record has been merged
After cleanup, check the mapping held by every connected system. If an integration still points to a removed record, it may interpret the missing destination as permission to create a replacement. That can recreate the duplicate you just removed.
Preserve a mapping from retired identifiers to the surviving record where the platform and integration support it. Define how archived and deleted records behave too. A deliberate deletion should not automatically be reversed by an old queued update.
Include a post-merge test: send an update using the former source relationship and inspect which destination changes. Then check its associations. This is especially useful when several systems maintain their own copy of the customer identity. The cleanup is complete only when those systems agree on the surviving relationship.
Decide which values survive before merging
Identity matching answers whether two records represent the same entity. Field retention answers what the merged record should contain. These are separate decisions.
The newest record is not automatically the most accurate. An enrichment job may have updated an old address yesterday, while a customer confirmed a different address last month. Record the source and meaning of important fields instead of choosing solely by modification date.
Before a bulk merge, agree on rules for ownership, current contact details, notes, open opportunities, service locations, and communication preferences. Preserve the more restrictive communication state while a conflict is reviewed; do not accidentally restart outreach because one duplicate has an empty preference field.
Check associated records too. A merge that leaves the contact looking correct but disconnects its open project has failed operationally. Preview representative pairs and preserve an export or recovery path appropriate to the platform before making irreversible changes.
For a one-time platform move, use the broader CRM data migration checklist. Recurring integration defects need ongoing controls in addition to migration cleanup.
Run a replay test before clearing the backlog
Use a test environment or controlled records that cannot trigger customer messages. The following cases form a practical acceptance set, not a claim that every integration needs the same implementation.
| Test | Expected result |
|---|---|
| Same event delivered three times | One intended activity, with repeats recorded |
| Response lost after successful creation | Existing result found; no second creation |
| Two workers create the same source entity | One destination identity or a visible conflict |
| Customer changes email | Existing mapped identity updated appropriately |
| Two people share a reception number | No automatic merge based on phone alone |
| Lookup service is unavailable | Work waits or alerts; failure is not treated as no match |
| Older update arrives after a newer one | Approved ordering policy preserves the correct state |
| Same company, separate service locations | Required location structure remains intact |
Check the destination after every test. A log message saying “deduplicated” does not prove the associations, fields, and downstream tasks are correct. Then restart the worker and repeat the relevant cases to check whether its memory survives a process restart.
Measure new errors, not just records removed
Track duplicate creation by entry channel after the fix. Keep the denominator visible: duplicates per new records or operations, depending on the object you are measuring.
For an illustrative example, suppose an import creates 20 duplicate companies among 500 intended companies. That is a 4% duplicate-creation rate for that run. If the next run creates two among 500, the rate is 0.4%. Those numbers are hypothetical; use verified outcomes from your own data.
A lower rate is insufficient if the new rule also combines unrelated accounts. Sample accepted matches and rejected candidates, track false merges, and confirm that valid new records still enter the CRM. Include time spent reviewing uncertain cases.
Keep cleanup volume separate from prevention quality. Removing 4,000 historic records can look impressive while the current integration keeps producing new ones. Conversely, a repaired integration may be valuable before the backlog is fully resolved.
What to request from your CRM partner
Ask for an origin analysis, the identity definition, a field-retention policy, a repeat-delivery test, and a named owner for exceptions. The deliverable should include enough evidence for your team to understand why a record was created or merged.
If the proposal begins and ends with a merge button, it addresses only part of the problem. A useful CRM integration project should leave the next import, retry, and changed customer detail easier to handle.
Start with one recent duplicate whose history you can inspect. Reproduce how it appeared, repair that path, and prove the repair with repeated and concurrent requests. Then work through the backlog with rules that have earned your trust.