Solved

A More Efficient Way To Write This Method

Posted on 2013-05-24
14
292 Views
Last Modified: 2013-05-24
Hi Experts!

Thanks for reading this.

There has to be a more efficient way of handing the string manipulation.
One thought is it use StringBuilder and passing it as ref..
Each time the method is called it creates the parameter (StrTemp), if I'm not mistaken.
The following method (ConvertStringWithSomeLogic) gets called 130K times and it takes 2+ hrs to complete..

Here's how the method is being called:

private static void InsertIntoDBTable(List<Model3> MdlCombine)
{
    SomeDB db = new SomeDB();
        foreach (Model3 m in MdlCombine)
        {
            Model3 st = new Model3();
            st.packagecodeid = ConvertStringWithSomeLogic(m.packagecodeid);
            st.STARTDATE = (m.STARTDATE);
            st.ENDDATE = (m.ENDDATE);
            db.Model3s.Add(st);
        }
        db.SaveChanges();
}

Open in new window


Here's the method being called:
      private static string ConvertStringWithSomeLogic(string StrTemp)
        {
            string strReturn = StrTemp;
            try
            {
                int IntFirstDash = 0;
                int IntSecondDash = 0;
                GetDashPos(ref IntFirstDash, ref IntSecondDash, StrTemp);

                string strFirstSegment = StrTemp.Substring(0, IntFirstDash);
                string strSecondSegment = StrTemp.Substring(IntFirstDash + 1, IntSecondDash - (IntFirstDash + 1));
                string strThirdSegment = StrTemp.Substring(IntSecondDash + 1, StrTemp.Length - (IntSecondDash + 1));

                //CONVERT THE FIRST, SECOND, THIRD SEGMENTS
                ConvertFirstSegment(ref strFirstSegment);
                ConvertSecondSegment(ref strSecondSegment);
                ConvertThirdSegment(ref strThirdSegment);

                strReturn = strFirstSegment + strSecondSegment + strThirdSegment;

                if ((strFirstSegment.Length < 4) || (strSecondSegment.Length < 3) || (strThirdSegment.Length < 1))
                {
                    strReturn = StrTemp;
                    //write msg to event log with StrTemp in it
                }
            }
            catch ....            
            return (strReturn);
        }

Open in new window


Here are the Convert Segment methods:

        private static void ConvertFirstSegment(ref string FirstSeg)
        {
            if (FirstSeg.Length == 4)
            {
                FirstSeg = "0" + FirstSeg;
            }
        }

        private static void ConvertSecondSegment(ref string SecondSeg)
        {
            if (SecondSeg.Length == 3)
            {
                SecondSeg = "0" + SecondSeg;
            }
        }

        private static void ConvertThirdSegment(ref string ThirdSeg)
        {
            if (ThirdSeg.Length == 1)
            {
                ThirdSeg = "0" + ThirdSeg;
            }
        }

Open in new window


TIA!!
0
Comment
Question by:allanau20
14 Comments
 
LVL 4

Expert Comment

by:jatkin
ID: 39193764
Hi,

I have just minimised the code down to this:

private static string ConvertStringWithSomeLogic(string StrTemp)
{
            string strReturn = StrTemp;
            string[] arr = StrTemp.Split('-');
            String firstSeg = (arr[0].Length == 4 ? "0" + arr[0] : arr[0]);
            String secondSeg = (arr[1].Length == 3 ? "0" + arr[1] : arr[1]);
            String thirdSeg = (arr[2].Length == 1 ? "0" + arr[2] : arr[2]);
            strReturn = firstSeg + secondSeg + thirdSeg;
            if ((firstSeg.Length < 4) || (secondSeg.Length < 3) || (thirdSeg.Length < 1))
            {
                strReturn = StrTemp;
                //write msg to event log with StrTemp in it
            }
            //   ...
            return strReturn;
}

Open in new window


and it executed the function 130k times in about 40ms...

James
0
 
LVL 5

Author Comment

by:allanau20
ID: 39193783
Thanks James; let me try it now ..
0
 
LVL 5

Author Comment

by:allanau20
ID: 39193815
I thought it was the method, but it's actually looping through the List of Model that's taking while to complete 130K records....
I'm going to catch some zzzzz....
0
 
LVL 62

Expert Comment

by:Fernando Soto
ID: 39194354
Hi allanau20;

If you are trying to process 130K records to the database using Entity Framework then the most likely cause of the long time is Entity Framework. When Entity Framework does create, update and delete records it sends a single command for each record in the 130K records. Is this what you are attempting to do?
0
 
LVL 62

Accepted Solution

by:
Fernando Soto earned 500 total points
ID: 39194496
Hi allanau20;

Here is a link to a article SqlBulkCopy for Generic List<T> (useful for Entity Framework & NHibernate) I have not tried this myself but I did read over the code and see no reason why it would not work.

Hope this helps your issue.
0
 
LVL 5

Author Comment

by:allanau20
ID: 39195033
Hey FernandoSoto.

Ok, I'll look into it in a few hours ...Thanks again.
0
 
LVL 38

Expert Comment

by:PaulHews
ID: 39195554
You may wish to turn off DetectChanges when adding large numbers of items to DBContext (I assume it's a DBContext?)

http://blog.oneunicorn.com/2012/03/12/secrets-of-detectchanges-part-3-switching-off-automatic-detectchanges/
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 5

Author Comment

by:allanau20
ID: 39195575
Hi PaulHews; yes, it's DBContext; I'll look into it. Thanks again.
0
 
LVL 62

Expert Comment

by:Fernando Soto
ID: 39195586
Turning off DetectChanges will not help in this case because most of your time will be spent sending a single record at a time to the SQL server and getting a response back for each of the 130K records. That is a whole lot of network congestion.
0
 
LVL 5

Author Comment

by:allanau20
ID: 39195713
I tried the DetectChanges and that didn't speed things up; thanks though PaulHews .
0
 
LVL 5

Author Comment

by:allanau20
ID: 39195715
I'm sooo stoked; I figured out how to applied the example. LOL..

It works; this is soo cool .. now it runs under a minute... Thanks again FernandoSoto ... lunch on me if you're in town...
0
 
LVL 5

Author Closing Comment

by:allanau20
ID: 39195720
awesome!
0
 
LVL 62

Expert Comment

by:Fernando Soto
ID: 39195747
Not a problem allanau20, glad I was able to help.

Hay, what town would that be. LOL.
0
 
LVL 5

Author Comment

by:allanau20
ID: 39195846
check your gmail...
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Creating an analog clock UserControl seems fairly straight forward.  It is, after all, essentially just a circle with several lines in it!  Two common approaches for rendering an analog clock typically involve either manually calculating points with…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now