SSIS between SQL and Oracle

Hi,

I need to extract rows from a MSSQL 2008 table and insert them into an Oracle database (SAP). I tried using Linked Server, but each transactions took between 4-16 seconds which was, of course, way too much. So I'm creating a SSIS.

I retrieve rows using a OLE DB Source on SQL Server and mapped the colums on a OLE DB Destination on Oracle (using Microsoft OLE DB Provider for Oracle). So far, it's fine. But I need to update an insert status in SQL, either fail or success. How can I do that?

Each rows inserted successfully in Oracle need to have its status updated in SQL. SSIS will be schedule to run at every 2 minutes, so new rows will be added in the SQL table between time recordset is retreive by the OLE DB Source query and the insertion in Oracle.

Anyone can help me please?
thanks for your time and help
Dominic34Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

rmm2001Commented:
Assuming that you are loading all of the rows at once and that they all succeed,..

Drag a sequence container to the canvas. Inside it put a data flow task. And then in there put your sql-> oracle logic. At success of the sequence container, drag an execute sql task (green line connecting the two) with an "UPDATE table SET Processed = 1 WHERE Processed IS NULL".

Doing this though you'll run into the problem if there are records being inserted into your source table at the same time you're trying to update - so potentially you could get records not being processed on the overlap.

To get around that I'd recommend creating two variables ("Now" and "lastupdated") and a ProcessedDate in your table.

At the end of your package, put the direct sql entry as "SELECT MAX(ProcessedDate) FROM yourtable" Selet "single row" as your output. On result set tab, select 0 as result set name and then on the drop down pick your LastUpdated variable name.

Then have the default value of "Now" be set to System::ContainerStartTime.

Go through your sequence container and send the rows from sql to oracle. Like before..At success of the sequence container, drag an execute sql task (green line connecting the two) with an "UPDATE table SET Processed = 1 WHERE ProcessedDate BETWEEN ? AND ?"

Set the first ? = User;:LastUpdated and the 2nd to User::Now

That will get your records between the time slot and allow you to process everything w/o overlaps.

One thing I'd recommend though - on your sql part - add in the "Processed" flag as part of your source query so you only grab the rows that aren't processed yet. so "SELECT stuff FROM table WHERE Processed IS NULL"
0
Dominic34Author Commented:
thanks a lot for your detailed reply.

Actually, in my Data Flow task, I have a OLE DB Source with the query:

SELECT '031' AS MANDT, NoBatch, NoLigne,
CAST(DATEPART(YEAR, GETDATE()) AS VARCHAR) + REPLACE(STR(DATEPART(MONTH, GETDATE()), 2, 0), ' ', '0') + REPLACE(STR(DATEPART(DAY, GETDATE()), 2, 0), ' ', '0') AS [Date],
REPLACE(STR(DATEPART(HOUR, GETDATE()), 2, 0), ' ', '0') + REPLACE(STR(DATEPART(MINUTE, GETDATE()), 2, 0), ' ', '0') + REPLACE(STR(DATEPART(SECOND, GETDATE()), 2, 0), ' ', '0') AS [Time],
' ' AS Statut
FROM dbo.TB_psapProductionSAP
WHERE Treated = 0

This query will return 0 to a few hundreds rows.

I have a OLE DB Destination on my Oracle connection and I mapped columns accordingly. Then I need to update a status on the source table in the SQL Server (TB_psapProductionSAP) wether the insert worked or failed, for each row. A row can be successfully inserted and the following row can fail. How can I manage that ?

thanks for your precious time
0
rmm2001Commented:
I'm sure there's a better way to do this..but the easy way would be to make the oledb destionation be a table or view access mode (that way it inserts the rows separately and not in bulk). Then on "Error Output" select "Redirect Row". This will redirect the entire row to the next transformation. Drag an oledb command under the oldeb destination and draw the red arrow to it. Then write in your update statement. "UPDATE TB_psapProductionSAP SET Status = 'Errored' WHERE ID  = ?" <-- That would require you to pass in your ID field to the source query ( it doesn't have to be mapped). You could also join on NoBatch & NoLinge  if those are unique fields. If they are it'd be "UPDATE TB_psapProductionSAP SET Status = 'Errored' where NoBatch = ? and NoLinge = ?" and you'd have to map the ?'s in the "column mapping" tab.

Now to record the success rows - I'd do what I posted first. Just update them to "Processed" where they're null. Which is taking the assumption that if they didn't error, they succeeded.

Have you taken into consideration with what would happen if the entire thing  catastrophically errored (yes I've gotten that exact error before) in the middle of execution and half of your rows are inserted and half aren't? If you restarted that it would lead to potential dupe entries. It's a rare case but one to think about.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Dominic34Author Commented:
thanks a lot for your help, I'll try it immediatly!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server 2008

From novice to tech pro — start learning today.