Perl script to check a website for profanity.

I have a perl script that checks a list of URLs (pulled from a database) to see if the link is active or dead.

I would like to also like to have the script check the content of the webpage for that URL, for inappropriate content (profanity, words of hate, etc...).

First, is this possible?
Second, how can it be done?

I'm not a PERL programmer, but I understand enough to get by. So the more detail you can provide the better.

Thanks in advance!!
LVL 2
verbal11Asked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
 
ozoConnect With a Mentor Commented:
use LWP::UserAgent;
open BANNED,"<bannedwords" or die $!;
my %banned;
chomp,$banned{lc $_}++ while <BANNED>;

my $ua = LWP::UserAgent->new;
while( <> ){
        chomp;
        my $response = $ua->get($_);
        if( !$response->is_success ){
            warn "$_ ".$response->status_line;
            next;
        }else{
              for my $w ( $response->content =~ /(\w+)/g ){
                  print "inappropriate $w in $_\n" if $banned{$w}
              }
        }
}
0
 
ozoCommented:
use Regexp::Common qw /profanity/;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
while( <> ){
        chomp;
        my $response = $ua->get($_);
        if( !$response->is_success ){
            warn "$_ ".$response->status_line;
            next;
        }else{
            $_ = $response->content;
            print "inappropriate $1\n" while /($RE{profanity}{contextual})/g;
        }
}
0
 
verbal11Author Commented:
Would you briefly break down for me what your script does?
Specifically:

... while /($RE{profanity}{contextual})/g;


Can I create a text file with a list of "banned" words that could be used in this script? How?
0
 
rwj04Commented:
i can help you compile the BANNED list.

tell me when you're ready to start.    i'm looking for my George Carlin collection now.

0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.