Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Load flat Files in Database

Posted on 2014-03-15
5
Medium Priority
?
101 Views
Last Modified: 2016-02-10
hi

can you please suggest me what are the fastest ways to load flat files to Database, If files size is over 1 GB.
best methods i know are
1. BCP
2. SSIS Load

do you have any more suggestions to this , which is faster and simpler.
0
Comment
Question by:BeyondBGCM
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 

Author Comment

by:BeyondBGCM
ID: 39931575
i have 200 pipe delimited files which i have to load in database in different tables(one to one mapping,which means one file is mapped to one database table.) , so i need a simpler and fastest solution . please suggest
0
 
LVL 66

Expert Comment

by:Jim Horn
ID: 39931782
I'm only familiar with SSIS (I have a hammer, everything looks like a nail), so I'll only speak to that.

Couple of possibilities:
1  Use a Raw File Source, which does not parse or translate the data, to optimize for speed.
2  Does this file represent a file you've loaded before, with a small subset of changes compared to the overall set?  If so, consider asking the source to send you a change-only file.
0
 

Author Comment

by:BeyondBGCM
ID: 39944811
1. I have 150 different files
2. I have to load them one by one , each mapped to one unique table in database.
3. I can use SSIS no issues.
0
 

Author Comment

by:BeyondBGCM
ID: 40635241
I havent received answer to this question yet .....
0
 
LVL 66

Accepted Solution

by:
Jim Horn earned 420 total points
ID: 40635306
>I have 150 different files .. I have to load them one by one , each mapped to one unique table in database.
You'll have to use an ETL tool such as SSIS, as pulling this off requires 150 different source to target mappings.
0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article we will learn how to fix  “Cannot install SQL Server 2014 Service Pack 2: Unable to install windows installer msi file” error ?
Windocks is an independent port of Docker's open source to Windows.   This article introduces the use of SQL Server in containers, with integrated support of SQL Server database cloning.
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question