Solved

Need to split one huge CSV file into 5 files

Posted on 2013-06-25
8
1,139 Views
Last Modified: 2016-02-11
Please help with the SSIS package, thanks
0
Comment
Question by:sqlcurious
8 Comments
 
LVL 4

Expert Comment

by:thelug
ID: 39275958
CSV Splitter.  Someone I knew in a previous "life" used it all the time.  Free utility.  Get it here:

http://erdconcepts.com/dbtoolbox.html
0
 

Author Comment

by:sqlcurious
ID: 39276169
Thelug, Thanks for the answer but I needto do it via SSIS, please suggest.
I know we could do using multicast or conditional split but not sure about the exact process
0
 
LVL 45

Expert Comment

by:aikimark
ID: 39322298
1. Are you splitting an input file or an output file?
2. Are you splitting based on row/records or by columns?
3. What kind of logic are you using to do the split?
4. When you say "huge", just how big is the file?

============
have you looked at Powershell?
http://www.codeproject.com/Articles/70319/Splitting-CSV-file-based-on-content-in-one-line-us
http://gallery.technet.microsoft.com/scriptcenter/PowerShell-Split-large-log-6f2c4da0
http://michielw.blogspot.com/2012/05/powershell-split-text-file-in-multiple.html

============
free CSV splitter to use while you are working on a complete solution
http://www.mydigitallife.info/download-free-csv-splitter-to-split-csv-files-easily/
0
Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

 
LVL 12

Accepted Solution

by:
Saurabh Bhadauria earned 500 total points
ID: 39322764
yes you can do it SSIS..
you have to use single  Flat file source and multiple  flat file destinations

if you want to split data based on row values then use Conditional Split ..
check  below link....
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server


If you want to Split data base on colums... then use multicast..
check below link..
http://stackoverflow.com/questions/6213847/using-multi-cast-transformation-for-destinations-with-different-columns 

Thanks,
Saurabh
0
 

Author Comment

by:sqlcurious
ID: 39334914
Thanks a lot Saurabh I will give it a try
0
 

Author Closing Comment

by:sqlcurious
ID: 39625802
thanks
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
A couple of weeks ago, my client requested me to implement a SSIS package that allows them to download their files from a FTP server and archives them. Microsoft SSIS is the powerful tool which allows us to proceed multiple files at same time even w…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Via a live example, show how to extract insert data into a SQL Server database table using the Import/Export option and Bulk Insert.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now