Solved

Need to split one huge CSV file into 5 files

Posted on 2013-06-25
8
1,240 Views
Last Modified: 2016-02-11
Please help with the SSIS package, thanks
0
Comment
Question by:sqlcurious
8 Comments
 
LVL 4

Expert Comment

by:thelug
ID: 39275958
CSV Splitter.  Someone I knew in a previous "life" used it all the time.  Free utility.  Get it here:

http://erdconcepts.com/dbtoolbox.html
0
 

Author Comment

by:sqlcurious
ID: 39276169
Thelug, Thanks for the answer but I needto do it via SSIS, please suggest.
I know we could do using multicast or conditional split but not sure about the exact process
0
 
LVL 45

Expert Comment

by:aikimark
ID: 39322298
1. Are you splitting an input file or an output file?
2. Are you splitting based on row/records or by columns?
3. What kind of logic are you using to do the split?
4. When you say "huge", just how big is the file?

============
have you looked at Powershell?
http://www.codeproject.com/Articles/70319/Splitting-CSV-file-based-on-content-in-one-line-us
http://gallery.technet.microsoft.com/scriptcenter/PowerShell-Split-large-log-6f2c4da0
http://michielw.blogspot.com/2012/05/powershell-split-text-file-in-multiple.html

============
free CSV splitter to use while you are working on a complete solution
http://www.mydigitallife.info/download-free-csv-splitter-to-split-csv-files-easily/
0
Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

 
LVL 12

Accepted Solution

by:
Saurabh Bhadauria earned 500 total points
ID: 39322764
yes you can do it SSIS..
you have to use single  Flat file source and multiple  flat file destinations

if you want to split data based on row values then use Conditional Split ..
check  below link....
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server


If you want to Split data base on colums... then use multicast..
check below link..
http://stackoverflow.com/questions/6213847/using-multi-cast-transformation-for-destinations-with-different-columns 

Thanks,
Saurabh
0
 

Author Comment

by:sqlcurious
ID: 39334914
Thanks a lot Saurabh I will give it a try
0
 

Author Closing Comment

by:sqlcurious
ID: 39625802
thanks
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to leverage one TLS certificate to encrypt Microsoft SQL traffic and Remote Desktop Services, versus creating multiple tickets for the same server.
Here's a requirements document template for an integration project (also known as Extract-Transform-Load or ETL) based on my development experience as an SQL Server Information Services (SSIS) developer over the years.
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

735 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question