[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now

x
?
Solved

How to check files recursively from multiple directories and get latest file in bash script

Posted on 2016-10-04
6
Medium Priority
?
182 Views
Last Modified: 2016-10-08
HI Experts,
How to check files recursively and get the latest file from the highest numbered directory.

Scenario:
1. We will have numbered directories 11110, 11114, etc.
2. Each above directory will have sub directories called changed and bkp
3. Each changed directory will have set of files...file1.txt, file2.txt, file3.txt etc. (bkp directory should not be considered in this logic)
4. We need to get latest numbered files in a output.txt

Based on the attached sample zip file below is the output
/tmp/test/11110/changed/file2.txt
/tmp/test/11114/changed/file5.txt
/tmp/test/11130/changed/file1.txt
/tmp/test/11130/changed/file3.txt
/tmp/test/11130/changed/file4.txt

Please help me with bash funciton where it accepts the directory path e.g /tmp/test/
0
Comment
Question by:enthuguy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 

Author Comment

by:enthuguy
ID: 41827616
0
 
LVL 7

Expert Comment

by:huacat
ID: 41828420
Try below command:
please remember changed the ./test to your path
for l in `find ./test -type d -name "changed"`; do find $l -type f -name "file*" | sort -r | head -1; done

Open in new window

0
 

Author Comment

by:enthuguy
ID: 41834624
hi Huacat,
could you help with this

Below is the out from your suggestion
./test/11110/changed/file5.txt
./test/11114/changed/file5.txt
./test/11130/changed/file4.txt

Based on the attached sample zip file below is the output
/tmp/test/11110/changed/file2.txt
/tmp/test/11114/changed/file5.txt
/tmp/test/11130/changed/file1.txt
/tmp/test/11130/changed/file3.txt
/tmp/test/11130/changed/file4.txt



Below is the structure of test directory
+--- test
|   +--- 11110
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   +--- 11114
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   +--- 11130
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 7

Expert Comment

by:huacat
ID: 41834673
@enthuguy, Im a bit of confuse the "latested numbered" file now.
Can you explain more about it?
How 11110 folder return file2.txt
How 11114 folder return file5.txt
How 11130 folder return file1.txt and file3.txt and file4.txt
0
 

Author Comment

by:enthuguy
ID: 41834679
sorry if I had confused you.

latest numbered
11110
11114
11130

from above example. Since file1.txt is available in all three directories...we need to get the file1.txt from 11130

since file2.txt is available only under 11110 We should consider the file under 11110

basically if same file available in all 3 we should pick the latest numbered directory in this 11130

Thanks
0
 
LVL 7

Accepted Solution

by:
huacat earned 2000 total points
ID: 41834762
Please run below code at test folder, you save it as a shell script to accept a argument.
folders=`find . -type d -name "changed" | sort`
names=`find ./ -name "*.txt" -follow -exec basename "{}" \; | sort | uniq`
for f in $names; do
   for d in $folders; do
     if [ -e $d/$f ]; then
       last=$d/$f
     fi
   done
   echo $last
done;

Open in new window

0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question