Advertisement

02.15.2007 at 03:26PM PST, ID: 22392945
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!

9.8

Optimization needed on string parsing

Asked by bhess1 in MS SQL Server

Tags: , , , ,

Below is a fraction of a stored procedure I am writing to parse out some non-normalized data into a more normal form.  

What we have are data records containing two specific fields:  MMYString (varchar 255) and MMYLongString (text).  These hold strings containing the make, model, and year of vehicles that the product can be used with.  The MMY strings look like this:

HONDA;CRF250R;2006|HONDA;CRF250R;2005|HONDA;CRF250R;2004

The information will be in MMYString unless the data length is more than 255 characters.  Then the data will be in MMYLongString, and no information will be in MMYString.  Note that we have data strings well over 30kb long for some cases.

I have created a routine that will strip the information out of a text field, and incorporated that into my code, a piece of which is below.  I am looking for any assistance in optimizing this routine, since I would like to run it nightly, and the process takes over an hour at this time.

------------------------------------------------
DECLARE @text table(txt text)


SET @separator = '|'

DECLARE c_Parse CURSOR FOR
      SELECT p.ProductID, lastModificationDate
      FROM SearchJoin sj
      INNER JOIN product P
            ON sj.productid = P.productId
      WHERE MMYExists = 'Y'
            AND lastmodificationdate >= @Cutoff
      ORDER BY p.ProductID

OPEN c_Parse
FETCH NEXT FROM c_Parse INTO @PID, @LMD

WHILE @@FETCH_STATUS = 0
BEGIN
      SELECT @WorkStr = ISNULL(MMYString, '') FROM searchJoin WHERE productid = @Pid
      IF @WorkStr = '' -- Data in mmylongstring
      BEGIN
            INSERT INTO @text
            SELECT MMYLongString AS Txt
            FROM SearchJoin sj
            WHERE sj.ProductID = @PID

            SELECT @i = 1,
                  @len = datalength(txt),
                  @curpos = 1
            FROM @text

            WHILE @curpos <= @len  -- Parse out a text field of arbitrary length in the
            BEGIN
                  select @lastpos=@curpos
                        ,@patindex=patindex('%'+@separator+'%',substring(txt,@lastpos+1,101))
                        ,@patindex=case @patindex when 0 then @len-@lastpos+1 else @patindex END
                        ,@val=substring(txt,@lastpos,@patindex)
                        ,@curpos=@lastpos+@patindex+1
                  FROM @text
            
                  If @Val LIKE '_%;_%;[1-2][0-9][0-9][0-9]' AND @val not like '%;%;%;%'
                  BEGIN
                        Set @Ptr = charindex(';',@val)
                        Set @Make = Left(@Val, @Ptr - 1)
                        SET @Ptr2 = CHARINDEX(';', @val, @ptr + 1)
                        Set @Model = Substring(@Val, @Ptr+1, @Ptr2 - (@ptr + 1))
                        Set @Year = SUBSTRING(@val, @Ptr2 + 1, LEN(@Val))
                  
                        SET @NewID = 0

                        SELECT @NewID = MMYID
                        FROM MMY
                        WHERE Make = @Make
                              AND Model = @Model
                              AND mmy.Year = @Year

                        IF IsNull(@NewID, 0) = 0
                        BEGIN
                        
                              INSERT INTO MMY (Make, Model, Year)
                              VALUES (@Make, @Model, @Year)
                  
                        END
                        IF NOT EXISTS (SELECT * FROM #Product_MMY WHERE ProductID = @PID AND MMYID = @NewID)
                              INSERT INTO #Product_mmy (ProductID, MMYID, LastModificationdate)
                              VALUES (@PID, @NewID, @LMD)
                  
                  END
            END

            DELETE
            FROM @Text
      END
      ELSE
      BEGIN
            WHILE LEN(@WorkStr)>0
            BEGIN
                  Set @Ptr = CHARINDEX(@Separator, @WorkStr, @Ptr2)
                  IF @Ptr = 0
                  BEGIN
                        SET @Val = @WorkStr
                        SET @WorkStr = ''
                  END
                  ELSE
                  BEGIN
                        SET @Val = LEFT(@WorkStr, @ptr-1)
                        SET @WorkStr = SUBSTRING(@WorkStr, @Ptr+1, LEN(@WorkStr))
                  END
                  Set @Make = Left(@Val, @Ptr - 1)
                  SET @Ptr2 = CHARINDEX(';', @val, @ptr + 1)
                  Set @Model = Substring(@Val, @Ptr+1, @Ptr2 - (@ptr + 1))
                  Set @Year = SUBSTRING(@val, @Ptr2 + 1, LEN(@Val))
                  
                  SET @NewID = 0

                  SELECT @NewID = MMYID
                  FROM MMY
                  WHERE Make = @Make
                        AND Model = @Model
                        AND mmy.Year = @Year

                  IF IsNull(@NewID, 0) = 0
                  BEGIN
                        
                        INSERT INTO MMY (Make, Model, Year)
                        VALUES (@Make, @Model, @Year)
                  
                  END
                  IF NOT EXISTS (SELECT * FROM #Product_MMY WHERE ProductID = @PID AND MMYID = @NewID)
                        INSERT INTO #Product_mmy (ProductID, MMYID, LastModificationdate)
                        VALUES (@PID, @NewID, @LMD)
            END
      END
      
      FETCH NEXT FROM c_Parse INTO @PID, @LMD
ENDStart Free Trial
 
Loading Advertisement...
 
[+][-]02.15.2007 at 10:37PM PST, ID: 18546551

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 7-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]02.16.2007 at 10:31AM PST, ID: 18550683

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]02.16.2007 at 09:42PM PST, ID: 18554067

View this solution now by starting your 7-day free trial. Setting up your free trial is quick, easy, and secure. We will return you to this solution, unlocked, when you're done.

 

About this solution

Zone: MS SQL Server
Tags: 1, charindex, cursor, optimizing, parse
Sign Up Now!
Solution Provided By: nmcdermaid
Participating Experts: 1
Solution Grade: A
 
 
[+][-]02.20.2007 at 09:09AM PST, ID: 18572056

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
 
Loading Advertisement...
20080716-EE-VQP-32