Solved

Grep Limit

Posted on 2004-08-23
9
1,109 Views
Last Modified: 2008-03-06

Is there a way to use Grep and set a limit on how far on each line it will search for the string you want.

Ex:
I want to search for a string only if it is in the first 200 characters of the line.

0
Comment
Question by:bt707
  • 4
  • 3
  • 2
9 Comments
 
LVL 20

Expert Comment

by:tfewster
ID: 11876528
How about:
cut -c1-200 filename |grep pattern

The obvious problem is that it will only return the truncated line...but you could use the first pass to generate the a list of matching line numbers and a second pass to extract those (complete) lines, e.g.

for LINE in `grep -n 1 filename | awk -F: '{printf $1 " "}'`
do
  sed -n -s ${LINE}p filename
done

But that would be inefficient for large files, so maybe use awk instead...

awk '{if ( substr($0,1,200) ~ "PATTERN" ) {print $0}}'  filename
0
 

Author Comment

by:bt707
ID: 11876673
that looks like it could do just what i need, don't care about it being truncated, but I'm getting this error when trying to run it.



awk '{if ( substr($0,1,200) ~ "@my.domain.com rfc822" ) {print $0}}' logfile1
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: bailing out near line 1
0
 
LVL 23

Expert Comment

by:brettmjohnson
ID: 11876717
You could use the cut command first:

#!/bin/bash
# Must have at least two args: pattern file...

if [ $# -lt 2 ] ; then
cat <<END
usage: $0 pattern files
Looks for regex pattern in first 200 chars of each file.
END

else
  pattern=$1
  shift 1

  for f in "$@" ; do
      tempf=`basename $f`.$$
      grep -H -n ".*" $f | cut -d":" -f1,2 > $tempf
      cut -c1-20 $f | paste -d":" $tempf - | grep -e "$pattern"
      rm -f $tempf
  done
fi


0
 
LVL 23

Expert Comment

by:brettmjohnson
ID: 11876737
Oops, that should be   cut -c1-200  



0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 
LVL 20

Expert Comment

by:tfewster
ID: 11876962
bt707, if I recall correctly, you're running Solaris;  I've just tested it on Solaris 8 using nawk instead of basic awk, and nawk executes without errors:

nawk '{if ( substr($0,1,200) ~ "@my.domain.com rfc822" ) {print $0}}' logfile1
0
 
LVL 20

Expert Comment

by:tfewster
ID: 11877017
This simplified version might work with standard Solaris awk - But  I won't be able to test it until tomorrow now...
awk ' substr($0,1,200) ~ "@my.domain.com rfc822"  {print $0}' logfile1
0
 
LVL 23

Expert Comment

by:brettmjohnson
ID: 11877044
Posted again with minor corrections:

#!/bin/bash
# Must have at least two args: pattern file...
if [ $# -lt 2 ] ; then
cat <<END
usage: $0 pattern files
Looks for regex pattern in first 200 chars of each file.
END

else
  pattern=$1
  shift 1

  for f in "$@" ; do
      tempf=/tmp/`basename $f`.$$
      grep -H -n ".*" $f | cut -d":" -f1,2 > $tempf
      cut -c1-200 $f | paste -d":" $tempf - | grep -h -e "$pattern"
      rm -f $tempf
  done
fi
0
 
LVL 20

Accepted Solution

by:
tfewster earned 500 total points
ID: 11879351
Here you go: A simple version for a crippled awk ;-)
awk ' substr($0,1,200) ~ /@my.domain.com.rfc822/  ' logfile1

And to answer your next question: Yes - the pattern can be passed as a variable ;-)

PATTERN="@my.domain.com.rfc822"
awk 'substr($0,1,200) ~ /'$PATTERN'/ ' logfile1
0
 

Author Comment

by:bt707
ID: 11879923
tfewster, thanks for all of your great answers, i used this last one and works great, just what I needed, haven't had time to try the other ones but sure they are all good ones.


Thanks again to All:
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

When you do backups in the Solaris Operating System, the file system must be inactive. Otherwise, the output may be inconsistent. A file system is inactive when it's unmounted or it's write-locked by the operating system. Although the fssnap utility…
Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now