Solved

Data migration using ADO.NET from a webservice

Posted on 2010-08-19
8
390 Views
Last Modified: 2012-05-10
Hi I would like to migrate data from a web service to oracle database using ADO.NET and I want to it be done in a faster way . Any help will be appreciated. I cannot do bulkcopy since I cannot access the source database, I can just get collection objects which need to be udpated in the database in efficient way (C#)

Please suggest... waiting for your reply.....
0
Comment
Question by:goldmaxhi
  • 4
  • 4
8 Comments
 
LVL 16

Expert Comment

by:kris_per
ID: 33480209

Something like this:
using (OracleConnection connection = new OracleConnection(connectionString))
            {
                connection.Open();

                foreach(MyObject m in myCollection)
                {
                    // first col is int and other two cols are text type
                    string queryString = @"INSERT INTO Table1 (col1, col2, col3) 
                                           values ({0}, '{1}', '{2}')";
                    queryString = string.Format(queryString,
                                                m.Field1,
                                                m.Field2,
                                                m.Field3);

                    OracleCommand command = new OracleCommand(queryString);
                    command.Connection = connection;                    
                    command.ExecuteNonQuery();
                }
            }

Open in new window

0
 

Author Comment

by:goldmaxhi
ID: 33480273
Hi Kris,

Thanks for the quick response... Can we use threads for this to make it even faster something like producer /consumer model...??
0
 

Author Comment

by:goldmaxhi
ID: 33480283

I am talking about the order for million records....
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 16

Expert Comment

by:kris_per
ID: 33480492

Threads are mainly to provide better user experience/responsive user interface and speed is not guaranteed by using threads (as execution in cpu is switched between different threads/apps by time-slicing)

Thread can be used for the above code...one option is - all of the above can be placed in a thread which will insert row after row...

If you mean the producer /consumer model for each row, though it is possible, I am not sure about the efficiency as it would involve one connection per task. Creating and closing a connection for each task/row wont be an efficient way...again db servers will have max limit for connections (sql server has max 100 connections at a time by default)...If you think about creating one connection and using it in multiple threads..well, i am not sure how that would work...

I will go for the first option of having a thread which will have all of the above code to loop thro collection and insert rows..simple and efficient, i think may be faster than having multiple threads trying to insert...


0
 
LVL 16

Expert Comment

by:kris_per
ID: 33480525

Another improvement that will increase the efficiency is to use query string with parameters, so that it can be pre-compiled using OracleCommand.Prepare method, which I am sure will increase the speed...

0
 

Author Comment

by:goldmaxhi
ID: 33480556
Kris... thank you very much for the answer....so nice of you...
0
 
LVL 16

Accepted Solution

by:
kris_per earned 500 total points
ID: 33480584

Thanks.....

With command.Prepare method, code will be as shown below...I am sure this will be more efficient than the prev code
using (OracleConnection connection = new OracleConnection(connectionString))
            {
                connection.Open();

                // first col is int and other two cols are text type
                string queryString = @"INSERT INTO Table1 (col1, col2, col3) 
                                           values (@val1, @val2, @val3)";

                OracleCommand command = new OracleCommand(queryString);
                command.Connection = connection;
                command.Parameters.Add("@val1", SqlDbType.Int);
                command.Parameters.Add("@val2", SqlDbType.Text);
                command.Parameters.Add("@val3", SqlDbType.Text);
                command.Prepare();

                foreach(MyObject m in myCollection)
                {                    
                    command.Parameters["@val1"].Value = m.Field1;                    
                    command.Parameters["@val2"].Value = m.Field2;                    
                    command.Parameters["@val3"].Value = m.Field3;

                    command.ExecuteNonQuery();
                }
            }

Open in new window

0
 

Author Closing Comment

by:goldmaxhi
ID: 33599816
Kris was very helpful in answering the question.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
ODBC in excel 2016 in Windows 10 via VBA 16 75
PrintingFoundIt(Almost!(Damn!)) 1 27
create a nested synonym 4 25
C# LINQ ForEach() question 6 28
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This video shows syntax for various backup options while discussing how the different basic backup types work.  It explains how to take full backups, incremental level 0 backups, incremental level 1 backups in both differential and cumulative mode a…
This video shows how to configure and send email from and Oracle database using both UTL_SMTP and UTL_MAIL, as well as comparing UTL_SMTP to a manual SMTP conversation with a mail server.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question