Solved

How to check files recursively from multiple directories and get latest file in bash script

Posted on 2016-10-04
6
119 Views
Last Modified: 2016-10-08
HI Experts,
How to check files recursively and get the latest file from the highest numbered directory.

Scenario:
1. We will have numbered directories 11110, 11114, etc.
2. Each above directory will have sub directories called changed and bkp
3. Each changed directory will have set of files...file1.txt, file2.txt, file3.txt etc. (bkp directory should not be considered in this logic)
4. We need to get latest numbered files in a output.txt

Based on the attached sample zip file below is the output
/tmp/test/11110/changed/file2.txt
/tmp/test/11114/changed/file5.txt
/tmp/test/11130/changed/file1.txt
/tmp/test/11130/changed/file3.txt
/tmp/test/11130/changed/file4.txt

Please help me with bash funciton where it accepts the directory path e.g /tmp/test/
0
Comment
Question by:enthuguy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 

Author Comment

by:enthuguy
ID: 41827616
0
 
LVL 6

Expert Comment

by:huacat
ID: 41828420
Try below command:
please remember changed the ./test to your path
for l in `find ./test -type d -name "changed"`; do find $l -type f -name "file*" | sort -r | head -1; done

Open in new window

0
 

Author Comment

by:enthuguy
ID: 41834624
hi Huacat,
could you help with this

Below is the out from your suggestion
./test/11110/changed/file5.txt
./test/11114/changed/file5.txt
./test/11130/changed/file4.txt

Based on the attached sample zip file below is the output
/tmp/test/11110/changed/file2.txt
/tmp/test/11114/changed/file5.txt
/tmp/test/11130/changed/file1.txt
/tmp/test/11130/changed/file3.txt
/tmp/test/11130/changed/file4.txt



Below is the structure of test directory
+--- test
|   +--- 11110
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   +--- 11114
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   |   +--- file5.txt
|   +--- 11130
|   |   +--- bkp
|   |   |   +--- file1.txt
|   |   |   +--- file2.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
|   |   +--- changed
|   |   |   +--- file1.txt
|   |   |   +--- file3.txt
|   |   |   +--- file4.txt
0
Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

 
LVL 6

Expert Comment

by:huacat
ID: 41834673
@enthuguy, Im a bit of confuse the "latested numbered" file now.
Can you explain more about it?
How 11110 folder return file2.txt
How 11114 folder return file5.txt
How 11130 folder return file1.txt and file3.txt and file4.txt
0
 

Author Comment

by:enthuguy
ID: 41834679
sorry if I had confused you.

latest numbered
11110
11114
11130

from above example. Since file1.txt is available in all three directories...we need to get the file1.txt from 11130

since file2.txt is available only under 11110 We should consider the file under 11110

basically if same file available in all 3 we should pick the latest numbered directory in this 11130

Thanks
0
 
LVL 6

Accepted Solution

by:
huacat earned 500 total points
ID: 41834762
Please run below code at test folder, you save it as a shell script to accept a argument.
folders=`find . -type d -name "changed" | sort`
names=`find ./ -name "*.txt" -follow -exec basename "{}" \; | sort | uniq`
for f in $names; do
   for d in $folders; do
     if [ -e $d/$f ]; then
       last=$d/$f
     fi
   done
   echo $last
done;

Open in new window

0

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question