Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

sorting csv files

Posted on 2008-10-21
5
Medium Priority
?
372 Views
Last Modified: 2013-12-27
I have a csv files which is produced dynamically
based on first record I need to sort the data and make different csvfiles

for example, in the file below,I need three different files with the records starting with
ACCNT_REVN_GROWTH, ACCNT_SURVEY_TYPE, ACCOUNT_ASSIGN_ROLE

ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All
ACCNT_SURVEY_TYPE,1,1,English-American,,1,Y,Y,N,All
ACCNT_SURVEY_TYPE,2,2,English-American,,2,Y,Y,N,All
ACCNT_SURVEY_TYPE,3,3,English-American,,3,Y,Y,N,All
ACCNT_SURVEY_TYPE,4,4,English-American,,4,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Primary,Primary,English-American,,1,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Secondary,Secondary,English-American,,2,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Tertiary,Tertiary,English-American,,3,Y,Y,N,All

exmple output

file1.csv

ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All

file2.csv

ACCNT_SURVEY_TYPE,1,1,English-American,,1,Y,Y,N,All
ACCNT_SURVEY_TYPE,2,2,English-American,,2,Y,Y,N,All
ACCNT_SURVEY_TYPE,3,3,English-American,,3,Y,Y,N,All
ACCNT_SURVEY_TYPE,4,4,English-American,,4,Y,Y,N,All

file3.csv

ACCOUNT_ASSIGN_ROLE,Primary,Primary,English-American,,1,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Secondary,Secondary,English-American,,2,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Tertiary,Tertiary,English-American,,3,Y,Y,N,All
0
Comment
Question by:conversekid
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 20

Assisted Solution

by:edster9999
edster9999 earned 2000 total points
ID: 22767428
Here is a script that splits them and names the files after that first field :


#!/bin/bash
 
cat test1.txt |while read line; do
        filename=`echo $line | awk -F, '{print $1}'`
        echo "$line" >> $filename.csv
done

Open in new window

0
 
LVL 20

Assisted Solution

by:edster9999
edster9999 earned 2000 total points
ID: 22767447
$ ls *.csv
ACCNT_REVN_GROWTH.csv  
ACCNT_SURVEY_TYPE.csv  
ACCOUNT_ASSIGN_ROLE.csv


example output file :

$ cat ACCNT_REVN_GROWTH.csv
ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All
0
 

Author Comment

by:conversekid
ID: 22767798
Great! your expertise is really appreciated....Its working but I am getting -ambiguous redirect- ..Is there any way we can avoid it?
0
 
LVL 20

Accepted Solution

by:
edster9999 earned 2000 total points
ID: 22767885
I based it on the file you gave.  
Are there other lines or blank line or anything like that ?

you could add in an echo line to show each line so you can spot which ones give errors and then put them up here and I'll adjust the file to fit that.

#!/bin/bash
 
cat test1.txt |while read line; do
        echo "Testing - $line"
        filename=`echo $line | awk -F, '{print $1}'`
        echo "$line" >> $filename.csv
done

Open in new window

0
 

Author Comment

by:conversekid
ID: 22768170
Hey Thats fine..As long as it works..Because the files are randomly generated everyday..
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question