Updating Very large table in SSIS

I have a very large table with over 200 million rows.  I also have a huge SSIS package that brings in data from a staging table, usually in 20million record clips.

Basically, I have all the conditions written where if a record comes in and the customer exists (by using a phone number as a unique record), I send it either down an insert (SQL server dest.)  or update path (OLE DB Command).  I want to have the newest data, so each record has an OptInDate column data stamp.  if there is a match on phone AND the optindate is newer, then I want to update...if it isn't newer, then do nothing.  IF the phone doesn't have a match, then it does an insert.

I have all of this working great (logically speaking).  Well, I hit a batch of data (about 60 million rows) that has much newer opt in dates, but it is customers we already have in the DB....so it will be an update condition.

Well, I was doing about 20million records processing per hour (which were only inserts).  Well, as soon as started doing an update...it went to a crawl...like 100,000 records per hour.

update is a simple update customer set col_a= ?, col_b? where ID = ?

Everything is indexed I can seek records very fast.

I have tried doing a delete where ID? and then directing it over to the insert side again...but really, I had the same performance.

Typically, if I get 20million records in, about 25% of that will be new data (inserts) the rest will be updates.

I have thought about using the merge join to process the updates (which i will store in a staging table).  How well will that work with something this big?  

Thanks for your help and I hope I explained it well..