Solved

sorting csv files

Posted on 2008-10-21
5
366 Views
Last Modified: 2013-12-27
I have a csv files which is produced dynamically
based on first record I need to sort the data and make different csvfiles

for example, in the file below,I need three different files with the records starting with
ACCNT_REVN_GROWTH, ACCNT_SURVEY_TYPE, ACCOUNT_ASSIGN_ROLE

ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All
ACCNT_SURVEY_TYPE,1,1,English-American,,1,Y,Y,N,All
ACCNT_SURVEY_TYPE,2,2,English-American,,2,Y,Y,N,All
ACCNT_SURVEY_TYPE,3,3,English-American,,3,Y,Y,N,All
ACCNT_SURVEY_TYPE,4,4,English-American,,4,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Primary,Primary,English-American,,1,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Secondary,Secondary,English-American,,2,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Tertiary,Tertiary,English-American,,3,Y,Y,N,All

exmple output

file1.csv

ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All

file2.csv

ACCNT_SURVEY_TYPE,1,1,English-American,,1,Y,Y,N,All
ACCNT_SURVEY_TYPE,2,2,English-American,,2,Y,Y,N,All
ACCNT_SURVEY_TYPE,3,3,English-American,,3,Y,Y,N,All
ACCNT_SURVEY_TYPE,4,4,English-American,,4,Y,Y,N,All

file3.csv

ACCOUNT_ASSIGN_ROLE,Primary,Primary,English-American,,1,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Secondary,Secondary,English-American,,2,Y,Y,N,All
ACCOUNT_ASSIGN_ROLE,Tertiary,Tertiary,English-American,,3,Y,Y,N,All
0
Comment
Question by:conversekid
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 20

Assisted Solution

by:edster9999
edster9999 earned 500 total points
ID: 22767428
Here is a script that splits them and names the files after that first field :


#!/bin/bash
 
cat test1.txt |while read line; do
        filename=`echo $line | awk -F, '{print $1}'`
        echo "$line" >> $filename.csv
done

Open in new window

0
 
LVL 20

Assisted Solution

by:edster9999
edster9999 earned 500 total points
ID: 22767447
$ ls *.csv
ACCNT_REVN_GROWTH.csv  
ACCNT_SURVEY_TYPE.csv  
ACCOUNT_ASSIGN_ROLE.csv


example output file :

$ cat ACCNT_REVN_GROWTH.csv
ACCNT_REVN_GROWTH,5 - 10%,5 - 10%,English-American,,3,Y,Y,N,All
ACCNT_REVN_GROWTH,10 - 15%,10 - 15%,English-American,,4,Y,Y,N,All
ACCNT_REVN_GROWTH,15 - 20%,15 - 20%,English-American,,5,Y,Y,N,All
ACCNT_REVN_GROWTH,> 20%,>20%,English-American,,6,Y,Y,N,All
0
 

Author Comment

by:conversekid
ID: 22767798
Great! your expertise is really appreciated....Its working but I am getting -ambiguous redirect- ..Is there any way we can avoid it?
0
 
LVL 20

Accepted Solution

by:
edster9999 earned 500 total points
ID: 22767885
I based it on the file you gave.  
Are there other lines or blank line or anything like that ?

you could add in an echo line to show each line so you can spot which ones give errors and then put them up here and I'll adjust the file to fit that.

#!/bin/bash
 
cat test1.txt |while read line; do
        echo "Testing - $line"
        filename=`echo $line | awk -F, '{print $1}'`
        echo "$line" >> $filename.csv
done

Open in new window

0
 

Author Comment

by:conversekid
ID: 22768170
Hey Thats fine..As long as it works..Because the files are randomly generated everyday..
0

Featured Post

[Webinar] How Hackers Steal Your Credentials

Do You Know How Hackers Steal Your Credentials? Join us and Skyport Systems to learn how hackers steal your credentials and why Active Directory must be secure to stop them. Thursday, July 13, 2017 10:00 A.M. PDT

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses

622 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question