Solved

Regular Expression Question

Posted on 2011-09-15
9
327 Views
Last Modified: 2012-05-12
I was given this as a solution to a problem of needing to remove blank lines from a file.
cat create_user.sql | egrep -i '[a-z0-9\-]'
The way i'm understanding i should get everything a thru z and 0 thru 9 plus the underscore _.
I got all this plus quotes double and single ' " and semicolon ; all of which is fine.  I just don't understand how i got these characters in addition to the ones in the regular expression.
0
Comment
Question by:xoxomos
  • 4
  • 4
9 Comments
 
LVL 35

Expert Comment

by:Terry Woods
Comment Utility
That command excludes lines without an alphabetic, numeric or hyphen character. It would exclude a line with only other characters, such as just ; on it for example.

If you want to exclude only blank lines, this looks good to me, using egrep:

cat create_user.sql |egrep -v '^$'
0
 
LVL 35

Expert Comment

by:Terry Woods
Comment Utility
Or, a nice alternative technique:

perl -pe 's/^\s*$//' create_user.sql   #outputs the file (like cat) without blank lines

And if you're happy with the result, run:

perl -i.bak -pe 's/^\s*$//' create_user.sql   #copy the original with a .bak extension, then commit the change
0
 

Author Comment

by:xoxomos
Comment Utility
Well yes and no.  Your explanation why cat create_user.sql | egrep -i '[a-z0-9\-]'   worked to eliminate blank lines sounds correct to me.

The alternative you gave to get rid of blank lines in the file does not work.

oracle@bb7b:/u05/datapump_directory$ cat create_user.sql | egrep -v '^$';
-- new object type path is: DATABASE_EXPORT/SYS_USER/USER
 ALTER USER "SYS" IDENTIFIED BY VALUES 'BBFEC2ABA14349B0'
      TEMPORARY TABLESPACE "TEMP";
 
-- new object type path is: DATABASE_EXPORT/SCHEMA/USER
 ALTER USER "SYSTEM" IDENTIFIED BY VALUES 'E3C232DAD21E66AE'
      DEFAULT TABLESPACE "TOOLS"
      TEMPORARY TABLESPACE "TEMP";
 
 CREATE USER "OUTLN" IDENTIFIED BY VALUES '4A3BA55E08595C81'
      TEMPORARY TABLESPACE "TEMP";
0
 
LVL 35

Expert Comment

by:Terry Woods
Comment Utility
cat create_user.sql | egrep -v '^$';

will only work if there are no whitespace characters on the line. This worked for me with whitespace though:

cat create_user.sql | egrep -v '^[[:space:]]*$';
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 37

Expert Comment

by:Gerwin Jansen
Comment Utility
To filter empy lines and lines contain spaces or tabs and create a new file:
grep -v '^[[:space:]]*$' create_user.sql > create_user_clean.sql

Open in new window

0
 

Author Comment

by:xoxomos
Comment Utility
In the original question http://www.experts-exchange.com/OS/Unix/Q_27308970.html i had tried to use the ^$ pattern to get rid of blank lines and it did not work.  Still don't know why not. :-(
0
 
LVL 35

Accepted Solution

by:
Terry Woods earned 250 total points
Comment Utility
The pattern ^$ matches only when the start of line (^) is next to the end of the line ($). If there's a space in between, it doesn't match.
0
 

Author Comment

by:xoxomos
Comment Utility

This does not get rid of blank lines either in this particular file anyhow.
The solution arnold gave me does work however.
http://www.experts-exchange.com/OS/Unix/Q_27308970.html
oracle@bb7b:/u05/datapump_directory$ grep -v '^[[:space:]]*$' create_user.sql > create_user_clean.sql
oracle@bb7b:/u05/datapump_directory$ vi *clean*
"create_user_clean.sql" 88 lines, 3161 characters
-- new object type path is: DATABASE_EXPORT/SYS_USER/USER
 ALTER USER "SYS" IDENTIFIED BY VALUES 'BBFEC2ABA14349B0'
      TEMPORARY TABLESPACE "TEMP";

-- new object type path is: DATABASE_EXPORT/SCHEMA/USER
 ALTER USER "SYSTEM" IDENTIFIED BY VALUES 'E3C232DAD21E66AE'
      DEFAULT TABLESPACE "TOOLS"
      TEMPORARY TABLESPACE "TEMP";

 CREATE USER "OUTLN" IDENTIFIED BY VALUES '4A3BA55E08595C81'
      TEMPORARY TABLESPACE "TEMP";

 CREATE USER "TRACESVR" IDENTIFIED BY VALUES 'F9DA8977092B7B81'
      DEFAULT TABLESPACE "SMALL_DEFAULT"
      TEMPORARY TABLESPACE "TEMP";

 CREATE USER "BBADMIN" IDENTIFIED BY VALUES '35DE13A3ED9CDD95'
      DEFAULT TABLESPACE "BBADMIN_DATA"
      TEMPORARY TABLESPACE "TEMP";
0
 

Author Closing Comment

by:xoxomos
Comment Utility
the perl works too.

That sounds like it might be the explanation, but when I set list in vi, i don't see anything between the beginning and $

- new object type path is: DATABASE_EXPORT/SYS_USER/USER$
 ALTER USER "SYS" IDENTIFIED BY VALUES 'BBFEC2ABA14349B0'$
      TEMPORARY TABLESPACE "TEMP";$
 $
-- new object type path is: DATABASE_EXPORT/SCHEMA/USER$
 ALTER USER "SYSTEM" IDENTIFIED BY VALUES 'E3C232DAD21E66AE'$
      DEFAULT TABLESPACE "TOOLS"$
      TEMPORARY TABLESPACE "TEMP";$
 $
 CREATE USER "OUTLN" IDENTIFIED BY VALUES '4A3BA55E08595C81'$
      TEMPORARY TABLESPACE "TEMP";$
 $
 CREATE USER "TRACESVR" IDENTIFIED BY VALUES 'F9DA8977092B7B81'$
      DEFAULT TABLESPACE "SMALL_DEFAULT"$
      TEMPORARY TABLESPACE "TEMP";$
 $
 CREATE USER "BBADMIN" IDENTIFIED BY VALUES '35DE13A3ED9CDD95'$
      DEFAULT TABLESPACE "BBADMIN_DATA"$
      TEMPORARY TABLESPACE "TEMP";$
 $
 CREATE USER "BB_BB60_STATS" IDENTIFIED BY VALUES 'DD5B4DC833EE258E'$
      DEFAULT TABLESPACE "BB_BB60_STATS_DATA"$
      TEMPORARY TABLESPACE "TEMP";$
 $
 CREATE USER "BB_BB60" IDENTIFIED BY VALUES '7EFB15396EAF1168'$
      DEFAULT TABLESPACE "BB_BB60_DATA"$
      TEMPORARY TABLESPACE "TEMP";$
 $
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
Installing FreeBSD… FreeBSD is a darling of an operating system. The stability and usability make it a clear choice for servers and desktops (for the cunning). Savvy?  The Ports collection makes available every popular FOSS application and packag…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now