Avatar of Newton
Newton
 asked on

Python : how to remove duplicated row from output.

Hi,
I got 2 files i.e File1 and File2, as shown below

File1
A | APPLE
B | ORANGE

File2
A | 10
B | 15
D | 20
A | 10

I need following output
Output 1
A | APPLE | 10
B | ORANGE | 15

But I am getting this below output.
A | APPLE | 10
B | ORANGE | 15
A | APPLE | 10

How can I remove the duplicate rows from the output and direct only the duplicate output to a new file.

My code is as follow

Import pandas as pd
df1 = pd.read_csv('file1.txt', sep='|')
df2 = pd.read_csv('file2.txt', sep='|')
Merge12 = pd.merge(df1, df2, how='left', on='A')
Merge12.to_csv('output.txt')
Python

Avatar of undefined
Last Comment
Subodh Tiwari (Neeraj)

8/22/2022 - Mon
Subodh Tiwari (Neeraj)

You may remove duplicate rows either from the df1 and df2 first and then merge them or remove duplicate rows from the resultant dataframe.


Merge12.drop_duplicates(keep='first', inplace=True)

Open in new window

Newton

ASKER
Below code worked.

Import pandas as pd
df1 = pd.read_csv('file1.txt', sep='|')
df2 = pd.read_csv('file2.txt', sep='|')
df2.drop_duplicates(keep='first',inplace=True)
Merge12 = pd.merge(df1, df2, how='left', on='A')
Merge12.to_csv('output.txt')

Now I want to write only the duplicated row to a new file, is below code is coreect way of doing?

Import pandas as pd
df1 = pd.read_csv('file1.txt', sep='|')
df2 = pd.read_csv('file2.txt', sep='|')
df2.drop_duplicates(keep='first',inplace=True)
Merge12 = pd.merge(df1, df2, how='left', on='A')
Merge12.to_csv('output.txt')

df3 = pd.read_csv('file.txt', sep='|')
df3.drop_duplicates(keep='first', inplace=False)
df3.to_csv('duplicatedrow.txt')
ASKER CERTIFIED SOLUTION
Subodh Tiwari (Neeraj)

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Newton

ASKER
Thank You Neeraj.
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
Subodh Tiwari (Neeraj)

You're welcome Newton!