Multiple Text Files INTO one - awk

we have two type of file

1. Type One  file name starts with T or K for ex. T01 or K17  or K55 or T02 it should has any 2 digit after T or K and ends with .txt
sample first type :

00:01:E8:D6:53:37: -> Port 1000
00:02:A5:4F:26:75: -> Port 15
00:04:AC:E3:E8:49: -> Port 8
00:0C:29:12:26:90: -> Port 1000
00:0C:29:4C:58:DE: -> Port 23
00:0C:29:63:C1:D4: -> Port 6
00:0C:29:96:26:FE: -> Port 26
00:0C:29:A9:87:02: -> Port 1000
00:0C:29:C2:4E:E7: -> Port 1000
00:0C:29:DE:4E:E5: -> Port 26
00:0C:29:F2:71:DD: -> Port 26
00:0E:7F:EC:1D:8E: -> Port 23
00:12:A9:C2:E0:A0: -> Port 1000
00:1B:21:2F:65:F4: -> Port 1
00:1B:21:2F:69:31: -> Port 24
00:1B:21:2F:69:3A: -> Port 9
00:1B:21:2F:69:47: -> Port 4
00:1B:21:9C:03:D1: -> Port 17
00:1B:21:C6:58:25: -> Port 28
00:25:61:45:1E:40: -> Port 1000
00:25:90:95:7A:6E: -> Port 1000
00:25:90:A4:3A:74: -> Port 1000
00:25:90:A4:3A:82: -> Port 1000
00:25:90:A4:40:2F: -> Port 1000
00:25:90:A4:41:5E: -> Port 1000
00:25:90:A4:41:6E: -> Port 1000
00:25:90:A4:56:20: -> Port 1000
00:25:90:A8:97:2F: -> Port 1000
00:25:90:A8:97:42: -> Port 1000
00:25:90:A8:97:43: -> Port 1000

Open in new window

We want to collect all this files into one but will adding each line the name of file without .txt

for ex.
00:25:90:A8:97:42: -> Port 1000  -> K17
00:25:90:A8:97:43: -> Port 1000  -> K16
... etc

2.  Second type of file : --> 00:18:6E:37:CF:28 --> 00:01:E8:D6:53:37 --> 00:01:E8:D6:53:37 --> 00:50:56:BE:70:D8 --> 00:04:AC:E3:E8:49 --> 00:04:AC:E3:E8:49 --> 02:D0:68:12:4B:CC --> 90:2B:34:9D:53:CB --> 90:2B:34:9D:53:CB --> 90:2B:34:A0:42:F3 --> 90:2B:34:A0:42:F3 --> E0:69:95:2E:90:A4 --> E0:69:95:2E:90:A4 --> 90:2B:34:A0:42:F3 --> E0:69:95:2E:90:A4 --> B8:AC:6F:97:82:6F --> 00:04:AC:E3:E8:49 --> 00:50:56:BE:36:C1 --> 00:50:56:96:50:FA --> 00:50:56:96:03:8B --> 00:50:56:96:03:C3 --> 00:50:56:96:03:8B --> 00:50:56:96:03:8B --> 00:50:56:96:03:8B --> 00:50:56:96:03:C3 --> 00:50:56:96:25:99 --> 00:50:56:96:03:C3 --> 00:50:56:96:03:8B --> 00:50:56:BE:D0:49 --> 00:50:56:BE:D0:49 --> 00:50:56:BE:D0:49

Open in new window

it will add to the top of the created text file this with adding --> Router to the end of each line.

Who is Participating?

Improve company productivity with a Business Account.Sign Up

farzanjConnect With a Mentor Commented:
To prepend, I would use the second  command first

cat routerresult.txt | gawk '{print $0" --> Router"}' > single.txt

grep ^ [KT][0-9][0-9].txt /dev/null | gawk '{print substr($0,9)" -> "substr($0,1,3)}'  >> single.txt
What does the second type of file start with--what would be the possible names of second type?
3XLcomAuthor Commented:

after complete the process of collecting into one we should delete


there should be the only file of collected.
A proven path to a career in data science

At Springboard, we know how to get you a job in data science. With Springboard’s Data Science Career Track, you’ll master data science  with a curriculum built by industry experts. You’ll work on real projects, and get 1-on-1 mentorship from a data scientist.

SurranoSystem EngineerCommented:
Dear 3XLcom,

As for type 1, try this:
grep ^ [KT][0-9][0-9].txt /dev/null | gawk '{print substr($0,9)" -> "substr($0,1,3)}'

Open in new window

As for type 2, I'm not sure I get your question properly but try this:

cat (...) | gawk '{print $0" --> Router"}'

Open in new window

If you mean the two file types are *both* called e.g. K22.txt or even the records are intermixed, with the difference being in record format:
- first record format: mac " -> Port " portno
- second record format: ip " --> " mac
(notice the different arrows!)
then try this:

grep ^ [KT][0-9][0-9].txt /dev/null | gawk '
  $2=="->" {print substr($0,9)" -> "substr($0,1,3)}
  $2=="-->" {print substr($0,9)" --> Router"}'

Open in new window

3XLcomAuthor Commented:
That worked :
grep ^ [KT][0-9][0-9].txt /dev/null | gawk '{print substr($0,9)" -> "substr($0,1,3)}'  > single.txt

and this is also work

cat routerresult.txt | gawk '{print $0" --> Router"}' > secondsingle.txt

i want to combine both to one second single will be the prepended to the single.txt

and i want to be removed all the other .txt files in the directory.

that is all
Instead of
cat routerresult.txt | gawk '{print $0" --> Router"}' > secondsingle.txt

cat routerresult.txt | gawk '{print $0" --> Router"}' >> single.txt


ls | grep -v 'single.txt' | xargs rm -f *txt
SurranoSystem EngineerCommented:
Good catch, but this way it'll delete all txt files, including single.txt. Try this instead:
ls *txt | grep -v 'single.txt' | xargs rm -f

Open in new window

or how about doing the simple way:
rm [KT][0-9][0-9]*txt routerresult.txt

Open in new window


Da!  Good.

Could you explain
grep ^ [KT][0-9][0-9].txt /dev/null expression please?

I mean ^ and /dev/null part to read file names.  Regex part is easy.
3XLcomAuthor Commented:
Thank you so much
Hi 3XLcom,

Thank you for the points but Surrano has mainly solved this problem.  It would have been nicer to split points between me and him.
3XLcomAuthor Commented:
Dear farzans ;

I have one more question :

so if he/she help me there i should give more points because in this question you helped me totally. And please check out the other too thanks
SurranoSystem EngineerCommented:
grep ^ is a trick to match all lines (even empty ones)

grep ... /dev/null is a trick to force printing of filename in front of line.
If you have only one file listed on the command line e.g.
grep regex K01.txt
then grep won't prefix the lines matched with "K01.txt:"

If you have more than one file e.g.
grep regex K01.txt /dev/null
then grep will prefix the lines. Add to this that /dev/null is always zero length, i.e. it contains 0 lines => no artifacts produced.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.