Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Compiling a list of leaf paths in website

Posted on 2003-02-21
14
Medium Priority
?
314 Views
Last Modified: 2010-04-21
How would I compile a list of all the leaf (children) paths that exist on a website (PERL or Ksh script). For example:

if my site contained this structure off of HTDOCS:

/public/test/
/public/test/test1a/
/public/test/test1b/
/public/test/test2/
/public/test/test3/apples/
/public/test/test3/oranges/

I would want the full paths of only the leafs:

/public/test/test1a/
/public/test/test1b/
/public/test/test2/
/public/test/test3/apples/
/public/test/test3/oranges/

0
Comment
Question by:894359
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 2
  • +4
14 Comments
 

Author Comment

by:894359
ID: 7996245
I'll post an additional 500 to make this a 1000 point question to whomever provides me with the correct answer!
0
 
LVL 38

Expert Comment

by:yuzh
ID: 7998706
Sorry I don't understand what do you try to achieve.

Do you want to pull all the subdir under /public/test/
in to the env var PATH

If it is the case, here's what you can do about it:

#!/bin/ksh
TMPLIST=/tmp/mydirlis
find /public/test -type d -print > $TMPLIST
NUMDIR=`cat $TMPLIST | wc -l`

#get rid of the perent dir from the list
NUMDIR=`expr $NUMDIR -1 `

tail -${NUMDIR} $TMPLIST > ${TMPLIST}.tmp
mv ${TMPLIST}.tmp $TMPLIST

#now put all the dir into PATH

for dir in `cat $TMPLIST ` ; do
   PATH=${PATH}:${dir}
done
export PATH

#do something else
exit
# END OF SCRIPT

IF THIS IS WHAT YOU WANT 500 POINT IS MORE THAN ENOUGH.

IF NOT, PLEASE GIVE AN GOOD EXAMPLE ANOUT WHAT YOU WANT TO DO.

cheers!

yuzh

0
 
LVL 21

Expert Comment

by:tfewster
ID: 8000071
`find` would actually return:

/public/test
/public/test/test1a
/public/test/test1b
/public/test/test2
/public/test/test3
/public/test/test3/apples
/public/test/test3/oranges

I assume you don't want /public/test OR /public/test/test3 because they have subdirectories (i.e. they're not "leaves")
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 21

Expert Comment

by:tfewster
ID: 8000473
I know nothing about webservers, but assuming only the "leaf" directories contain web pages, how about this?

find /public/test -name "*.html" -exec dirname {} \; > dirs.txt
sort -u dirs.txt > dirs2.txt
sed 's/$/\// dirs2.txt > dirs.txt
rm dirs2.txt
0
 
LVL 21

Expert Comment

by:tfewster
ID: 8000506
Alternatively, this script seems to work for all the combinations I could think of:

find /public/test -type d  > data.txt
# Tag the "/" on the end; This is vital for when comparing the names
sed 's/$/\//' data.txt > data2.txt
# Sort in reverse order, so a parent follows its children
sort -r data2.txt > data3.txt
# Now compare each line in the file with the previous line;
# We know the first line will be a child, so print it anyway.
awk -v pdir=`head -1 data3.txt` 'BEGIN {print pdir ; getline } {
# If the current line is in the previous line, it must be
# a parent of the previous line; The negation gives us the children
if ( pdir !~ $0 ) {print $0}
else { print $0 "=PARENT"} # Delete this line after testing
pdir=$0
}' data3.txt > data4.txt
sort data4.txt > data5.txt

# I've left all the temp files in, so you can check the output at each stage
0
 
LVL 1

Expert Comment

by:irisex
ID: 8004417
Do a script more or less like this:

#!/bin/ksh

# Run in the root dir of the website

# For each directory...
for j in $(find . -type d)
do
  # Get number of files it contains (long ls begins with -)
  numfiles=$(ls -l |grep "^-"|wc)
  if [ $numfiles -gt 0 ]
  then
    # If it contains any file, echo it
    echo $j
  fi
done

Give me my first accepted answer, pleeaasee :-)
0
 
LVL 38

Expert Comment

by:yuzh
ID: 8005554
some of the dir under the webserver document Root dir might not have any html file at all !!!
0
 
LVL 4

Expert Comment

by:Vinit Kain
ID: 8007493
try this

find ./|awk -F\/ '{
for(i=1;i<NF;i++)
{
printf $(i);
printf "/"
}
printf "\n"
}'|sort -u
0
 
LVL 1

Accepted Solution

by:
rob-g earned 2000 total points
ID: 8024973
I think that I understand your problem. I haven't tried all the script examples above, but the ones I have tried didn't answer your question as I understand it. Here's my answer in the form of a Korn shell script.

To run this for a specific directory hierarchy change $PWD or parameterise it.

========================================================
#!/bin/ksh
# This Korn shell script will list list all leaf directories
# whether they contain files or not.

find $PWD -type d|awk '{f=$0"\n"f}END{print f}'|awk '
BEGIN{d[""]=1;}
{
  f=0;
  for (i in d) {
       l0=length()+1;
          si=substr(i,0,l0)
    if (si == $0"/") {f=f+1; break;}
  }
     if (f == 0) d[$0]=1;
}
END{for (i in d) print i}'|sort
========================================================

0
 

Expert Comment

by:dolphin-gmbh
ID: 8028352
w/o temp files and w/o head:
find $PWD -type d|sort -r|awk ´BEGIN{getline;prev=$0;print $0}{if(prev !~$0){print $0} prev=$0}´
cheers, happy browsing
0
 
LVL 21

Expert Comment

by:tfewster
ID: 8029045
dolphin-gmbh, don't forget the sed command to tag the "/" on the end, or it will mess up on a directory structure like:
/public/test/test1
/public/test/test1a

rib-g's fascinating script (Took me a while to figure out that  awk '{f=$0"\n"f}END{print f}' == sort -r ;-) produces the same output as mine.

However, I'm still not clear on the definition of a "leaf" in website terminology; If /public/test/test3 in the initial example contained web pages as well as subdirectories, is it 1) a valid website structure and 2) a "leaf"? I guess the points will go to whoever understands HTDOCS...
0
 

Expert Comment

by:dolphin-gmbh
ID: 8029860
hm...i think it's correct to have
/public/test/test1
/public/test/test1a
because each is a valid directory. isn't it ?
what i understand is:
he searches for the deepest path w/o a subdirectory.
btw: my 'find' doesn't return a trailing '/' on dirs,
so i changed the testbed a little ;-)
this is my structure here (w/o trailing '/'):
/public/test/
/public/test/test1a/
/public/test/test1b/
/public/test/test2/
/public/test/test3/apples/
/public/test/test3/oranges/
and i get this:
/public/test/test3/oranges
/public/test/test3/apples
/public/test/test2
/public/test/test1b
/public/test/test1a
and of course....piping in a 'sort' gives the sorted result.
hm...may i'm wrong :-)
0
 

Expert Comment

by:dolphin-gmbh
ID: 8029879
uhhhhhh, i see....don't kill me :-)
0
 

Expert Comment

by:dolphin-gmbh
ID: 8029939
find $PWD -type d|sort -r|sed 's/$/\//'|awk 'BEGIN{getline;prev=$0;print $0}{if(prev !~$0){print $0} prev=$0}'|sort| sed 's/\/$//'

also killes the last '/'
uffff :-)
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Suggested Courses

704 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question