Solved

Load flat Files in Database

Posted on 2014-03-15
5
90 Views
Last Modified: 2016-02-10
hi

can you please suggest me what are the fastest ways to load flat files to Database, If files size is over 1 GB.
best methods i know are
1. BCP
2. SSIS Load

do you have any more suggestions to this , which is faster and simpler.
0
Comment
Question by:BeyondBGCM
  • 3
  • 2
5 Comments
 

Author Comment

by:BeyondBGCM
ID: 39931575
i have 200 pipe delimited files which i have to load in database in different tables(one to one mapping,which means one file is mapped to one database table.) , so i need a simpler and fastest solution . please suggest
0
 
LVL 65

Expert Comment

by:Jim Horn
ID: 39931782
I'm only familiar with SSIS (I have a hammer, everything looks like a nail), so I'll only speak to that.

Couple of possibilities:
1  Use a Raw File Source, which does not parse or translate the data, to optimize for speed.
2  Does this file represent a file you've loaded before, with a small subset of changes compared to the overall set?  If so, consider asking the source to send you a change-only file.
0
 

Author Comment

by:BeyondBGCM
ID: 39944811
1. I have 150 different files
2. I have to load them one by one , each mapped to one unique table in database.
3. I can use SSIS no issues.
0
 

Author Comment

by:BeyondBGCM
ID: 40635241
I havent received answer to this question yet .....
0
 
LVL 65

Accepted Solution

by:
Jim Horn earned 210 total points
ID: 40635306
>I have 150 different files .. I have to load them one by one , each mapped to one unique table in database.
You'll have to use an ETL tool such as SSIS, as pulling this off requires 150 different source to target mappings.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
Viewers will learn how to use the SELECT statement in SQL to return specific rows and columns, with various degrees of sorting and limits in place.
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question