Solved

Load flat Files in Database

Posted on 2014-03-15
5
99 Views
Last Modified: 2016-02-10
hi

can you please suggest me what are the fastest ways to load flat files to Database, If files size is over 1 GB.
best methods i know are
1. BCP
2. SSIS Load

do you have any more suggestions to this , which is faster and simpler.
0
Comment
Question by:BeyondBGCM
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 

Author Comment

by:BeyondBGCM
ID: 39931575
i have 200 pipe delimited files which i have to load in database in different tables(one to one mapping,which means one file is mapped to one database table.) , so i need a simpler and fastest solution . please suggest
0
 
LVL 66

Expert Comment

by:Jim Horn
ID: 39931782
I'm only familiar with SSIS (I have a hammer, everything looks like a nail), so I'll only speak to that.

Couple of possibilities:
1  Use a Raw File Source, which does not parse or translate the data, to optimize for speed.
2  Does this file represent a file you've loaded before, with a small subset of changes compared to the overall set?  If so, consider asking the source to send you a change-only file.
0
 

Author Comment

by:BeyondBGCM
ID: 39944811
1. I have 150 different files
2. I have to load them one by one , each mapped to one unique table in database.
3. I can use SSIS no issues.
0
 

Author Comment

by:BeyondBGCM
ID: 40635241
I havent received answer to this question yet .....
0
 
LVL 66

Accepted Solution

by:
Jim Horn earned 210 total points
ID: 40635306
>I have 150 different files .. I have to load them one by one , each mapped to one unique table in database.
You'll have to use an ETL tool such as SSIS, as pulling this off requires 150 different source to target mappings.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have a large data set and a SSIS package. How can I load this file in multi threading?
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question