Python : how to remove duplicated row from output.

You may remove duplicate rows either from the df1 and df2 first and then merge them or remove duplicate rows from the resultant dataframe.

Merge12.drop_duplicates(keep='first', inplace=True)

Select allOpen in new window

Below code worked.

Import pandas as pd
df1 = pd.read_csv('file1.txt', sep='|')
df2 = pd.read_csv('file2.txt', sep='|')
df2.drop_duplicates(keep='first',inplace=True)
Merge12 = pd.merge(df1, df2, how='left', on='A')
Merge12.to_csv('output.txt')

Now I want to write only the duplicated row to a new file, is below code is coreect way of doing?

Import pandas as pd
df1 = pd.read_csv('file1.txt', sep='|')
df2 = pd.read_csv('file2.txt', sep='|')
df2.drop_duplicates(keep='first',inplace=True)
Merge12 = pd.merge(df1, df2, how='left', on='A')
Merge12.to_csv('output.txt')

df3 = pd.read_csv('file.txt', sep='|')
df3.drop_duplicates(keep='first', inplace=False)
df3.to_csv('duplicatedrow.txt')

Thank You Neeraj.

You're welcome Newton!