• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 494
  • Last Modified:

Trying to get the string in a *.log file and search for that string name file throughout the folders and subfolders

How to grep the string from a file, then find that string name file throughout ./ and store it in a separate folder
0
Vijay kumar Mohanraj
Asked:
Vijay kumar Mohanraj
  • 18
  • 11
  • 3
  • +1
9 Solutions
 
tel2Commented:
Hi mail2vijay1982,

I'm struggling to understand what you mean.
For example:
- "find the string name file".  What does that mean?
- "store it in a separate folder"?  Store what, exactly?
Forgive me, Ingrish is only my first language.

Please explain your problem in detail, AND I suggest you provide some sample:
- Input data, and
- Expected results.
So we can clearly understand what you mean.

Thanks.
0
 
Monis MontherSystem ArchitectCommented:
to find the file that contained a specific string simply use the ( -l switch )

ex:  grep -rl "some words with spaces" /

-r : search recursively in the dir
-l : output file name containing the string

Now this would take a lot of time

0
 
Monis MontherSystem ArchitectCommented:
Oh sorry didnt read the title well enough, since you are looking under log files then it would simply be

grep -rl "some words with spaces" /var/log

Note: You must have the " " if the sentence has spaces
0
[Webinar] Improve your customer journey

A positive customer journey is important in attracting and retaining business. To improve this experience, you can use Google Maps APIs to increase checkout conversions, boost user engagement, and optimize order fulfillment. Learn how in this webinar presented by Dito.

 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
How to grep the string " *.xsl " from a file, then find that string named  file throughout ./  folders and sub folders. And store those files in a separate folder. Let it be shell script or commands, but commands would be more preferable as it is easy to execute right away without keeping my hand in the permissions..

The second question is to search for a string name "export" in all  .log file through all directories, sub directories and copy those  .log files which contains that string "export" to separate folder.
0
 
TintinCommented:
I'm interpreting the question as grepping out a string from a bunch of logs and then searching for filenames (not content) with that name.

If so, then


#!/bin/bash
grep string /some/dir/*.log | read read string
do
  find / -name "*string*" 
done

Open in new window

0
 
TintinCommented:
Ah, wrote my last post before I saw yours.  I was on the right track though.


#!/bin/bash
cd /to/some/path
grep ".xsl" file | while read string
do
  find . -name "$file" -type f | xargs -i mv {} /path/to/separate folder
done

Open in new window

0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
that string could contain name, like " *.xls"..is it possible..
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
i dont want to move, want to copy it actually...so i think cp applies for that right


#!/bin/bash
cd /to/some/path
grep ".xsl" file | while read string
do
  find . -name "$file" -type f | xargs -i cp {} /path/to/separate folder
done

Open in new window

0
 
TintinCommented:
Are you saying the string in the file you are grepping could be

*.xsl

0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
yes, so it can be anything   .xsl like,  abc.xsl or bcd.xsl,...
0
 
Monis MontherSystem ArchitectCommented:
If you have an * in the string put a \ before it

grep "\*.xls" filename
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
which consists in those .log file..
0
 
Monis MontherSystem ArchitectCommented:
Ok now I understand what you want.

Here you go I tested this and it works fine

 for i in $(grep ".xls" /root/file); do find / -name $i -exec cp {} /path/to/dir \;; done


Note there is a ; ; two semicolons after the \
0
 
Monis MontherSystem ArchitectCommented:
Now the second task is done in the way I pointed out in my first post by using the grep -rl
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small student: i have to use this command in very big Linux environment. I heard that using of exec with large storage files gives lots of errors. So my sources asked me to use xargs. But i am not sure about that. What do u think???
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small student: i have to use this command in very big Linux environment. I heard that using of exec with large storage files gives lots of errors. So my sources asked me to use xargs. But i am not sure about that. What do u think???
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
Tintin: i have tried a sample of ur code, but nogo, Can you check this...

BTW, this is for the first question,



#!/bin/bash
cd ./thelogfiles/
grep -il "/*.xls" *.log | while read string
do
  find . -name "$*.log" -type f | xargs -i cp {} ./extractedfiles/
done

Open in new window

0
 
Monis MontherSystem ArchitectCommented:
Well instead of exec you can use ok , it will prompt you for each copy and you can choose yes or no, but I think (Although not 100% sure) that the exec in my command is not the same as the bash exec its a find built in switch with the same name.

Try this and if you dont like the result you can Ctrl +C and nothing is harmed

 for i in $(grep ".xls" /path/to/file); do find / -name $i -ok cp {} /path/to/dir \;; done
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
small_student:I have tried this command along, no errors, but no output to...nothing in the /extractedfiles..


for i in $(grep ".xls" ./thelogfiles); do find ./Documents/ -name $i -ok cp {} ./extractedfiles \;; done

Any suggestions..
0
 
Monis MontherSystem ArchitectCommented:
Ok Lets route out the confusion

./thelogfiles    (this should be a file not a dir, if it is a dir then use grep -r ".xls"...etc)
./Documents  (This is where all the files are located and you are trying to seach in them, this was / in your first post)

./extractedfiles  (This is the location that the files will finally end up being copied to)

Note: Use full paths instead of relative paths for all three of the above

0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
small_student: Plz do check this line, giving warning msg...

for i in $(grep "/*.xls" ./thelogfiles/*); do find ./Documents/ -name $i -ok cp {} ./extractedfiles \;; done

the warning msg:

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.

 Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.
< cp ... ./Documents/see.xls > ?
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small_student:

./Documents  (let it be inside this folder or any sub folders inside it, it should search all directiores and      subdirectories inside ./Documents)

./thelogfiles/*.log ( would this help out)


for i in $(grep "/*.xls" ./thelogfiles/*.log); do find ./Documents/ -name $i -ok cp {} ./dis \;; done
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
sorry not ./dis

for i in $(grep "/*.xls" ./thelogfiles/*.log); do find ./Documents/ -name $i -ok cp {} ./extractedfiles \;; done
0
 
Monis MontherSystem ArchitectCommented:
1- No you cant do this, first off all you cant have a / to escape a wildcard like * you put \
2- dont use any * in anything here
3- type the command as follows

for i in $(grep -r ".xls" ./thelogfiles/); do find ./Documents/ -name $i -ok cp {} ./extractedfiles \;; done
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
small_student:

I have noticed one thing that using -exec, able to export at least one file to ./extractedfiles

With -ok, no output, ..nothing in the /extractedfiles..
but getting error msg as same mentions above,


the warning msg:

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.

 Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.
< cp ... ./Documents/see.xls > ?

0
 
Monis MontherSystem ArchitectCommented:
Please follow the tips in my last post and let me know what happens
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small_student: thats what iam been saying already, i tried this

for i in $(grep -r ".xls" ./thelogfiles/); do find ./Documents/ -name $i -ok cp {} ./extractedfiles \;; done

and getting this error msg,

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `./thelogfiles/something.log:File'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.

 Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `./thelogfiles/something.log:File''.
< cp ... ./Documents/see.xls > ?
0
 
Monis MontherSystem ArchitectCommented:
aha Ok I think I know what the problem is, the grep is getting more that a file name, its getting a full path, we need to filter it to the file names only.

Lets do a little test

I want to see the output of the grep part only, please run the following command and post its output here

grep -rl ".xls" /thelogfiles | head
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small_student:

Tried this,
grep -rl ".xls" ./thelogfiles | head

OUTPUT:

./thelogfiles/oh135.log
./thelogfiles/12the.log
./thelogfiles/233345t4w_456.log
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
sorry and this too,

./thelogfiles/something.log
0
 
Monis MontherSystem ArchitectCommented:
Oh sorry I made a mistake here, this output shows the files that actually have lines in them containing the .xls, what I wanted is the line itself.

Post the output of this

grep -r ".xls" ./thelogfiles | head

0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:

small_student:

Tried this,
grep -r ".xls" ./thelogfiles | head

the output

./thelogfiles/oh135.log:123.xls
./thelogfiles/12the.log:File name           456.xls
./thelogfiles/233345t4w_456.log:File name           789.xls
0
 
Vijay kumar MohanrajCloud ArchitechAuthor Commented:
for the second question, i sorted it out by my self with the support of internet..

for the first question the answer is still pending,

Code:
for i in $(grep -rh ".xls"  ./thelogfiles/) ; do find ./Documents/ -iname $i -print0 -exec cp {} ./extractedfiles/ \;; done

the "-h", removes the path and just shows the file name.
So now, iam getting the output in "extractedfiles" folder. Except for the null spaces Ex:this that.xls. I think i can sort it out.
 But if any one have solution for this null space, you are very well appreciated. And if any one can do this code with xargs instead of exec, do comment below...

I appreciate all the above experts for what they have done so far for helping me out, thanks
0

Featured Post

[Webinar] Improve your customer journey

A positive customer journey is important in attracting and retaining business. To improve this experience, you can use Google Maps APIs to increase checkout conversions, boost user engagement, and optimize order fulfillment. Learn how in this webinar presented by Dito.

  • 18
  • 11
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now