Solved

Trouble Importing Flat File Into MS SQL 2008

Posted on 2014-02-27
2
567 Views
Last Modified: 2014-02-27
Good morning!

I've got a very large pipe delimited flat file that I'm having difficulty importing into SQL.

Here's the situation:

The file is about 650MB (a few million records
The file is pipe delimited
I don't know the max lengths of the values in the fields
I tried importing the file into a table in temdb with all of the fields set to varchar(max) and I still get truncation errors in SQL.
I'm getting truncation errors on a specific column when I tell the import/export wizard to ignore truncation and even when I tell it to ignore the entire column

I've been able to open the file in a program called "Stat Transfer" (something we purchased along with a program called STATA - an SPSS competitor).  That reads about 4,400 records but, it seems to think those are all the records in the file.  

I suspect there's some kind of odd character around the end of that 4,400th record that's causing these issues.  Typically, I would search around in the text file and manually edit it to fix something like that but, this file is too big to open in notepad or notepad++.  

So what to do?  Any ideas as to how I might go about opening or editing the source file?  

Thanks in advance!
0
Comment
Question by:ttist25
2 Comments
 
LVL 11

Accepted Solution

by:
John_Vidmar earned 500 total points
ID: 39891801
There are free file-splitting utilities.  Break the file into smaller pieces, and then open the smaller file using Notepad++ to find out what's going on around that 4,400th record.
0
 
LVL 1

Author Comment

by:ttist25
ID: 39893243
Thanks John.  

Your method would definitely work so I'm awarding points but, I ended up using something called "File Query" by AgileSoftware.  It's a free trial and it was pretty useful so I may actually pay for it.  It opened the file right up and loaded all of the records.  I was able to query min.max lengths etc.  I'm still working to get it into MSSQL but, this worked well for opening and allowing edits.  

Thanks!
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Suggested Solutions

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
This article explains all about SQL Server Piecemeal Restore with examples in step by step manner.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now