duplicate leads treatment best practice in CRM application

Dear Experts

I understand there are two approaches to handle the duplicate records when same prospect enters the CRM application
approach 1) append duplicate prospect data to a single record approach 2) create duplicate record and mark it status as "Repeated/Duplicated"

approach1) if we go for append duplicate prospect data to a single record would it be possible to have a marketing analysis like current month how many leads entered into SugarCRM including the duplicate leads also would it be possible to understand prospect journey touch points through multi channel showing interest.
approach2)same prospect entering through multiple channels setting duplicate records to “repeated” status would be the most straightforward to implement and link it with the original lead may provide the marketing analysis like current month how many leads entered into SugarCRM including the duplicate leads and from which all campaigns and source.
if we go for approach two then the manual operation can be applied with the same treatment that is if the already existing prospect makes enquiry to the call centre then call centre user applies the same treatment that is copy/create the existing lead and mark the status as "Repeated" and attach it to the respective campaigns
Please suggest which approach in the above is  recommended and best practice , thanks in advance
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Kevin CrossChief Technology OfficerCommented:
Think your points above are good arguments for keeping lead sources separate but keep in mind if then want to see how many of your leads from a particular source ended up as accounts, you may lose visibility if you have a bunch of leads that never convert ... Given Sugar supports multi-select fields, I would make merge the records but keep multiple sources in list then when doing your counts, you can add extra for fields with multiple sources.  Think it stores it as ^value1^value2^ so you can parse it from SQL easily — at least hosting locally you can but suspect the cloud versions reporting can handle this also if not a calculated field.  With this approach, you have all versions of that lead able to track accurately that lead moved to an account, to a customer (won opportunity), et cetera.

Hope that helps!
D_wathiAuthor Commented:
Thanks Kevin for the reply.
 I have asked the software developer for customization that links a duplicate prospect record to the original prospect,
if same prospect comes in multiple times through the integrations that is before the lead is Converted to Opportunity record then all are marked as "Repeated/Duplicated" and linked to the orgin lead.
if same prospect comes in multiple times after the lead is converted to to Opportunity then all these leads still marked as "repeated/duplicated" and linked to the Contact record in the lead section.
 following condition considered as new lead
 if lead is Dead  i,e before Converted to opportunity and if after Converted and opportunity record sales stage found to be sale closed won OR sale closed lost in these cases new lead will be allowed to create as fresh lead I mean witout the "Repeated/Duplicated" Status but when try to convert NO Contact record will be allowed instead new opportunity can be created.

If we go with above approach we can measure the prospect coming through multiple channels and as well measure the markeitng campaign effectiveness,
request you suggestion please , thanks in advance.
Kevin CrossChief Technology OfficerCommented:
It sounds like you have your mind made up on a plan and ways to get around the challenges, so not sure you need much more here but for what it is worth I think managing the campaigns and maintaining multiple records for the same contact should be separate issues.  Campaigns table just links to an id, so there is nothing stopping you from having the same id (after you merge) show up in multiple places for reporting but think for your long-term sanity of maintaining contacts having mass amounts of duplicates will be trouble IMHO but it may be because all the companies I know using Sugar CRM or Salesforce ultimately want to track leads through beyond just accounts but into their business systems as customers.  Therefore, most will let their marketing automation bring in duplicates but on conversion they de-dupe the record.  If that just doesn't work for you, the other thing I know to be true is no two businesses operate exactly the same and I would go with your gut as long as you have done due diligence on what happens after conversion given lead is just one small part of a contacts life cyle (with luck).
OWASP Proactive Controls

Learn the most important control and control categories that every architect and developer should include in their projects.

D_wathiAuthor Commented:
So nice of you for such a great support. I just wanted to understand on what you suggest that becomes very important for as we have still not gone live. please suggest.
Kevin CrossChief Technology OfficerCommented:
I thought I was being clear, so please forgive me.  I have been suggesting you go with approach 1.  It is much easier to capture / count the duplicates for purposes of giving credit to marketing campaigns, sales people, et cetera than it is to manage later an instance cluttered with multiples.  The point of Sugar CRM is to give you better insight into the sales funnel, correct?  Therefore, think about your 1 lead giving you $10K opportunity... does this opportunity grow to $100K just because you repeated the same person 10 times?  If you want my opinion, it remains approach 1.  Almost had me convinced but then starts with customization to get the duplicates organized, so now you would be adding a modification to system that you have to pay for and worry about during upgrades; therefore, even from a financial perspective, I have to go back to approach 1.
D_wathiAuthor Commented:
Thanks Kevin. I understood but not one input from you, by default sugarcrm does to support out of the box on approach 1 this has to go for customization, the easiest to achieve approach2 ,
1. even when we go for approach2 I agree with you on manual user operation we can merge how about the integration leads pushed to Sugarcrm like web2lead, online chat, face book Ad leads,
2. lead would have moved to opportunity stage and now same prospect enters as lead how the merge can happen here, in this case should we have allow it through the integrations as "Repeated/Duplicated" and then manually merge
I am sorry hope iam not disturbing you too much
Kevin CrossChief Technology OfficerCommented:
You are not bothering me.  I just wasn't catching that you want an automated merge.  Sugar CRM most certainly supports merging the records but we choose to do this manually because humans can better differentiate John Smith, Llc from John Smith, Co; therefore, with that in mind, I see your challenge is not that you want to keep the duplicates but you need a way to tag them easily and approach 2 does that.  I was getting hung up on approach 1 stating "single record" as manual merge is in Sugar CRM 7.x with suggestions as soon as you save with records that match.  I do have the global search in place as well though, so not sure if you do not get the potential dupes messages without it.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Kevin CrossChief Technology OfficerCommented:
What version of Sugar are you using?
D_wathiAuthor Commented:
Hi Kevin

I am using 7.9 .1.0 , we had old process running,  now are migrating on new process this dedupe is main hence getting this to be the best. I also agree the approach 1 is good but it requires too much customization is what understood I mean when it has to be done through the automatic merge. your suggestion will be my decision
D_wathiAuthor Commented:
Hi Kevin
Thanks for your support just before closing this case, I request you support is it possible to hide the leads/opportunities which are of status "Repeated/Duplicated" so that when users  perform Global search they do not get the Repeated/Duplicated records of any module listed. please suggest can we achieve this please. thanks in advance.
Kevin CrossChief Technology OfficerCommented:
I am not sure you can hide them from Global search as that either searches module or it does not from my experience.  However, you can use a filter on the list view for Leads.  In addition, the Global search usually only indexes items with save from the UI.  If you are updating these via a SQL query to the new status, it doesn't count.  Further, when you are updating them, you could put something in the name like "[Repeat] Lead Name" so when they come up on Global search the users know to skip them right away.
D_wathiAuthor Commented:
Hi Kevin Cross
May I request iam i going in right direction your comments would be a great input to me please.
Kevin CrossChief Technology OfficerCommented:
See comment right above this one.  Global search is a module on|off system -- at least my experience with it.  The only times I have seen things not show up was when I did automation|data integration straight through SQL Server.  When I use API calls, the records show as if I saved from UI and as such Global Search picks them up for indexing.  I do not know of another way to hide them from Global Search because of that.  In other words, if your sync or activity acts through API|UI, the item will show up in the search.  If your customization is SQL-based, then it won't show up in the search unless someone goes into the record from within Sugar UI.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.