richsark
asked on
Search, sort, then output into csv format
Hello,
I need a way to look at one csv file (DNS-record.csv that looks like this. But it has a few thousand lines
File 2 named: Global-text.csv looks like this, and it has thousands of lines
What I wish it can do is look in file 1 and search in file 2 named Global-text.csv for a matching DNS name (column 1) to a matching txt record on the 4th row ( text*)
The Global-text.csv has every single txt to hostname in it. But Im only interested in what it finds from file 1"DNS-record.csv"
If found, then output the results named output.csv with the above format from ( file 2 Global-text.csv )
Is t here a way to shell script, perl maybe or even grep/awk? I am using a mac as well
I would appreciate any help
I need a way to look at one csv file (DNS-record.csv that looks like this. But it has a few thousand lines
DNS Name,IP Address,Text Record
dcdiag-test-record.corp.sark.com,1.1.1.1,
omniview.corp.sark.com,10.0.0.12,
omniviewwebprd.corp.sark.com,10.0.0.12,
ocrntovftp.corp.sark.com,10.0.0.14,
omniviewftpprd.corp.sark.com,10.0.0.14,
File 2 named: Global-text.csv looks like this, and it has thousands of lines
header-txtrecord,fqdn*,_new_fqdn,text*,_new_text,comment,disabled,ttl,view,EA-Site
txtrecord,dcdiag-test-record.corp.sark.com,,31f8d425a979fff1cbc064fedc6af8cd7f,,,FALSE,3600,Internal-RPZ,
txtrecord,omniviewwebprd.corp.sark.com,,00a151deec7b2c49fcb106daab945e376f,,,FALSE,3600,Internal-RPZ,
txtrecord,zbr4185224.ocr,,00444e8f6a21e53453f0e11a6d6f1e9f67,,,FALSE,3600,Internal-RPZ,
What I wish it can do is look in file 1 and search in file 2 named Global-text.csv for a matching DNS name (column 1) to a matching txt record on the 4th row ( text*)
The Global-text.csv has every single txt to hostname in it. But Im only interested in what it finds from file 1"DNS-record.csv"
If found, then output the results named output.csv with the above format from ( file 2 Global-text.csv )
Is t here a way to shell script, perl maybe or even grep/awk? I am using a mac as well
I would appreciate any help
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
...and a 3rd question:
Q3. How can you match the DNS Name to the text* column? The DNS Name seems to contain things like:
dcdiag-test-record.corp.sa rk.com
while the text* column seems to contain things like:
31f8d425a979fff1cbc064fedc 6af8cd7f
so they won't match.
Do you mean the DNS name should be matched against column 2, i.e. _new_fqdn,text*?
Q3. How can you match the DNS Name to the text* column? The DNS Name seems to contain things like:
dcdiag-test-record.corp.sa
while the text* column seems to contain things like:
31f8d425a979fff1cbc064fedc
so they won't match.
Do you mean the DNS name should be matched against column 2, i.e. _new_fqdn,text*?
ASKER
Hi tel 2:
Q1. When you say:
"...to a matching txt record on the 4th row ( text*)"
Yes
Q2. What is the expected output for your sample input data?
Same as file 2
@Bill Prew
I will try your awk too
Q1. When you say:
"...to a matching txt record on the 4th row ( text*)"
Yes
Q2. What is the expected output for your sample input data?
Same as file 2
header-txtrecord,fqdn*,_new_fqdn,text*,_new_text,comment,disabled,ttl,view,EA-Site
txtrecord,dcdiag-test-record.corp.sark.com,,31f8d425a979fff1cbc064fedc6af8cd7f,,,FALSE,3600,Internal-RPZ,
txtrecord,omniviewwebprd.corp.sark.com,,00a151deec7b2c49fcb106daab945e376f,,,FALSE,3600,Internal-RPZ,
txtrecord,zbr4185224.ocr,,00444e8f6a21e53453f0e11a6d6f1e9f67,,,FALSE,3600,Internal-RPZ,
@Bill Prew
I will try your awk too
Q4. If the expected output is the same as one of the input files, then why do you need a script? You already have what you need.
Please also answer Q3 in my 2nd post.
Please also answer Q3 in my 2nd post.
ASKER
Hi. The reason is I have a short list in my file one. Which i need it matched to the file 2 which has the master list per say. Once all the matching is done spit out a modified version of all the matches that looks like file 2.
In that case, the output will not be the same as file 2. It will have less records, so the sample output data you provided is incorrect. Please correct it. I suggest you always provide sample input data and matching expected output data in your original post, when you ask questions in future, so experts don't have to guess, and can confirm they have understood your requirements by looking at the output data, and comparing the output of their proposed solutions with it. This can also help experts to get past any errors you've made in your specifications, like those I have raised. So, you may get your correct solution faster, and it can save a lot of time for experts who are volunteering their precious time. So, please answer Q3 again.
And I'm still waiting for your answer to Q4.
Q5. Do you have the Unix/Linux "join" command on your Mac?
And I'm still waiting for your answer to Q4.
Q5. Do you have the Unix/Linux "join" command on your Mac?
Assuming the answers to my questions are what I'm guessing they are, and assuming you have the "join" command on your Mac, this should work:
#!/bin/bash
head -n1 DNS-record.csv >DNS-record.csv.srt # Store header
tail -n+2 DNS-record.csv | sort -t, -k1,1 >>DNS-record.csv.srt # Append sort of all except header
head -n1 Global-text.csv >Global-text.csv.srt # Store header
tail -n+2 Global-text.csv | sort -t, -k2,2 >>Global-text.csv.srt # Append sort of all except header
# Join on 1st file field 2 & 2nd file field 1, outputing columns 1-10 of 1st file only
join --header -t, -12 -21 -o1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,1.10 Global-text.csv.srt DNS-record.csv.srt >output.csv
rm DNS-record.csv.srt Global-text.csv.srt # Optional clean up
Here's the contents of output.csv after running the above:
header-txtrecord,fqdn*,_new_fqdn,text*,_new_text,comment,disabled,ttl,view,EA-Site
txtrecord,dcdiag-test-record.corp.sark.com,,31f8d425a979fff1cbc064fedc6af8cd7f,,,FALSE,3600,Internal-RPZ,
txtrecord,omniviewwebprd.corp.sark.com,,00a151deec7b2c49fcb106daab945e376f,,,FALSE,3600,Internal-RPZ,
But Bill's should basically do the same, and might even be slightly more efficient because his doesn't need to do any sorting, creation of temporary files, etc.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks guys as always for your expertise
Welcome, glad that was useful.
»bp
»bp
Q1. When you say:
"...to a matching txt record on the 4th row ( text*)"
do you mean this?:
"...to a matching txt record on the 4th column ( text*)"
Q2. What is the expected output for your sample input data?
tel2