troubleshooting Question

How to do join update between Pandas DataFrames based on a common column

Avatar of Sean Girgis
Sean GirgisFlag for United States of America asked on
* Datascience* pandasPython
5 Comments2 Solutions24 ViewsLast Modified:

I have a Pandas Dataframe that include many columns such as (Server, Region, types, no_of_cores, ....., app_id) and there is a numerical index for each row.

name that Dataframe left.

Say that Dataframe has 100k rows... about a 2000 of those rows has Nan in the app_id. 


I also have another DataFrame that will name it right it has only two columns (Server, appID)

There is about 10000 rows and in the right dataframe and all of them has values in appID.


How can we update the first dataframe (left) ... from Right dataframe updating the field 

right.app_id = left.appID having that Server is the same in both data frames and when only right.app_id is Nan



Here is a cope explanation.. I need to reach desired outcome. I am using python 3.78


import pandas as pd, numpy as np
right = pd.DataFrame(columns=['Server', 'region','no_of_cores', 'Memory', 'app_id' , 'notes']) 
right = right.append({'Server':'sevA', 'region':'East','no_of_cores':8, 'Memory':48, 'app_id':'101|102' , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevB', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':np.NaN , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevC', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'105' , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevD', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'105' , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevE', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'108' , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevF', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':np.NaN , 'notes':'some note'},ignore_index=True)
right =right.append({'Server':'sevG', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'102|107' , 'notes':'some note'},ignore_index=True)
print(right)

print('===================================================')
left = pd.DataFrame(columns=['Server','app_id' ]) 
left = left.append({'Server':'sevB', 'app_id':'101|102' },ignore_index=True)
left = left.append({'Server':'sevC', 'app_id':'101|102' },ignore_index=True)
left = left.append({'Server':'sevD', 'app_id': np.NaN },ignore_index=True)
left = left.append({'Server':'sevF', 'app_id': '143|144' },ignore_index=True)
left = left.append({'Server':'sevH', 'app_id': '115|117' },ignore_index=True)
print(left)
print('=================Desired Outcome====================================')
outcome = pd.DataFrame(columns=['Server', 'region','no_of_cores', 'Memory', 'app_id' , 'notes']) 
outcome = outcome.append({'Server':'sevA', 'region':'East','no_of_cores':8, 'Memory':48, 'app_id':'101|102' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevB', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'101|102' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevC', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'105' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevD', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'105' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevE', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'108' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevF', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'143|144' , 'notes':'some note'},ignore_index=True)
outcome =outcome.append({'Server':'sevG', 'region':'West','no_of_cores':8, 'Memory':72, 'app_id':'102|107' , 'notes':'some note'},ignore_index=True)
print(outcome)

Open in new window


ASKER CERTIFIED SOLUTION
NorieSenior Associate
Log in to continue reading
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform for $9.99/mo
View membership options
Unlock 2 Answers and 5 Comments.
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
See how we're fighting big data
The Value of Experts Exchange in My Daily IT Life

Experts Exchange (EE) has become my company's go-to resource to get answers. I've used EE to make decisions, solve problems and even save customers. OutagesIO has been a challenging project and... Keep reading >>

Mike

Owner of Outages.IO
Phoenix, Arizona, United States
Member Since 2016
Join a full scale community that combines the best parts of other tools into one platform.
Unlock 2 Answers and 5 Comments.
View membership options
“All of life is about relationships, and EE has made a virtual community a real community. It lifts everyone's boat.”
William Peck

Member since 2004