Solved

Load flat Files in Database

Posted on 2014-03-15
5
78 Views
Last Modified: 2016-02-10
hi

can you please suggest me what are the fastest ways to load flat files to Database, If files size is over 1 GB.
best methods i know are
1. BCP
2. SSIS Load

do you have any more suggestions to this , which is faster and simpler.
0
Comment
Question by:BeyondBGCM
  • 3
  • 2
5 Comments
 

Author Comment

by:BeyondBGCM
Comment Utility
i have 200 pipe delimited files which i have to load in database in different tables(one to one mapping,which means one file is mapped to one database table.) , so i need a simpler and fastest solution . please suggest
0
 
LVL 65

Expert Comment

by:Jim Horn
Comment Utility
I'm only familiar with SSIS (I have a hammer, everything looks like a nail), so I'll only speak to that.

Couple of possibilities:
1  Use a Raw File Source, which does not parse or translate the data, to optimize for speed.
2  Does this file represent a file you've loaded before, with a small subset of changes compared to the overall set?  If so, consider asking the source to send you a change-only file.
0
 

Author Comment

by:BeyondBGCM
Comment Utility
1. I have 150 different files
2. I have to load them one by one , each mapped to one unique table in database.
3. I can use SSIS no issues.
0
 

Author Comment

by:BeyondBGCM
Comment Utility
I havent received answer to this question yet .....
0
 
LVL 65

Accepted Solution

by:
Jim Horn earned 210 total points
Comment Utility
>I have 150 different files .. I have to load them one by one , each mapped to one unique table in database.
You'll have to use an ETL tool such as SSIS, as pulling this off requires 150 different source to target mappings.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Problem with SqlConnection 5 108
separate column 24 19
SQL Server memory Issue 7 73
SQL query to summarize items per month 5 23
From time to time, for debugging and troubleshooting your flow at run time you’ll need to check if the variable has the correct value or not, there’re several ways to check for the value of the variables inside the flow, You can add Break Points, a …
My client has a dictionary table. They're defining a list of standard naming convention. Now, they are requiring my team to provide us a mechanism how to match new incoming data with existing data in their system.
Via a live example, show how to extract insert data into a SQL Server database table using the Import/Export option and Bulk Insert.
Viewers will learn how to use the SELECT statement in SQL to return specific rows and columns, with various degrees of sorting and limits in place.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now