We help IT Professionals succeed at work.

Bulk Insert Row Terminator

T-Virus
T-Virus asked
on
733 Views
Last Modified: 2012-02-23
Hi,

i got a flat file which i would like to Bulk insert into a table

Flatfile Format

1,2,3,4,5/n
1,2,3,4/n
1/n

Open in new window


So the problem that i have is, that Bulk insert is only looking for the row terminator in the last column.

So the table would look like this:
col1 col2 col3 col4 col5
1      2     3      4     5
1      2     3      4     1

Instead of
col1 col2 col3 col4 col5
1      2     3      4     5
1      2     3      4    
1

Is there a way to give the row terminator a higher priority or something?
I guess i would have the same issue with xml format files.
Comment
Watch Question

Senior DBA
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Hmmm not the answer i was hoping for but hey we have to life with it.

I do know tools that where written in java that would work trough the exact same flatfile in less then 5sec. (2mil rows)

So in your opionion which way is the fastest?

Some Script that would do the job before Bulk inserting the data or
this Function? Even if the Function would take 30min to transform the date into a new table not faltfile funnily, there it would take just 5sec. But is it really the way to go with import export import?

CREATE FUNCTION dbo.fnSplit    (
          @Expression NVARCHAR(max)
        , @Delimiter  NVARCHAR(max)
        , @INDEX      INT
    )
    RETURNS NVARCHAR(max)
    AS
    BEGIN
        DECLARE @RETURN  NVARCHAR(max)
        DECLARE @Pos     INT
        DECLARE @PrevPos INT
        DECLARE @I       INT
       
        -- SELECT dbo.fnSplit('4.55.108.2','.', 2)
       
        IF @Expression IS NULL OR @Delimiter IS NULL OR LEN(@Delimiter) = 0 OR @INDEX < 1
            SET @RETURN = NULL
        ELSE IF @INDEX = 1 BEGIN
            SET @Pos = CHARINDEX(@Delimiter, @Expression, 1)
            IF @Pos > 0 SET @RETURN = LEFT(@Expression, @Pos - 1)
        END ELSE BEGIN
            SET @Pos = 0
            SET @I = 0
           
            WHILE (@Pos > 0 AND @I < @INDEX) OR @I = 0 BEGIN
                SET @PrevPos = @Pos
                SET @Pos = CHARINDEX(@Delimiter, @Expression, @Pos + LEN(@Delimiter))
               
                SET @I = @I + 1
            END
           
            IF @Pos = 0 AND @I = @INDEX
                SET @RETURN = SUBSTRING(@Expression, @PrevPos + LEN(@Delimiter), LEN(@Expression))
            ELSE IF @Pos = 0 AND @I <> @INDEX
                SET @RETURN = NULL
            ELSE
                SET @RETURN = SUBSTRING(@Expression, @PrevPos + LEN(@Delimiter), @Pos - @PrevPos - LEN(@Delimiter))
        END
       
        RETURN @RETURN
    END

Open in new window

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.