Solved

profanity filter from text file

Posted on 2004-08-06
7
1,218 Views
Last Modified: 2008-01-09
Hi there

I have a list of words in a text file line by line and I need to use a text file that contains line by line profanity words. Then use that to find in another file text file and delete that line if it contains any of the words from the other file.

For example say the profanity word list is

air
bee
call

the text file with words is

air tree with leaves
bee fore you go there
I like trees
Not before dinner
every time after call
be have

then all that is left is

I like tree
not before dinner
be have

Can you make sure that the removed lines are collpased so they are after each other so there are no gaps in the output text file

thanks
 
0
Comment
Question by:playstat
  • 3
  • 2
  • 2
7 Comments
 
LVL 3

Accepted Solution

by:
newaira earned 500 total points
ID: 11736089
try this:

<?php

//read lines of the bad word file into array
$badword = file('badwords.txt');

//remove newline character from each array value
for ($i=0; $i<count($badword); $i++)
  $badword[$i] = rtrim($badword[$i]);

//read lines of the file to check
$lines = file('listofwords.txt');

$count = count($lines);

for ($i=0; $i<$count; $i++) {
  for ($j=0; $j<count($badword);$j++) {
    if ( false != stristr($lines[$i], $badword[$j]) ) {
      unset($lines[$i]);
    }
  }
}

$lines = array_values($lines);
$lines = implode("", $lines);

$handle = fopen("newlistofwords.txt", "w");
fwrite($handle, $lines);
fclose($handle);

?>
0
 
LVL 33

Expert Comment

by:snoyes_jw
ID: 11736518
Bear in mind that this method will also remove lines that contain bad words inside other words.  For example,  with your sample word list, you'll lose lines containing "Blair", "recall", "heebee-jeebies".  If that's a concern, you'll want to use regular expressions to check for word boundaries around the bad word.
0
 
LVL 3

Expert Comment

by:newaira
ID: 11736608
Thanks for clarifying snoyes_jw.
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 

Author Comment

by:playstat
ID: 11737615
could you add them please it would be most helpful

thanks
0
 
LVL 33

Expert Comment

by:snoyes_jw
ID: 11737726
replace the line
if ( false != stristr($lines[$i], $badword[$j]) ) {
with these two:
$pattern = "/(?i)\b" . $badword[$j] . "\b/";
if (preg_match($pattern, $lines[i])) {

I think...
0
 

Author Comment

by:playstat
ID: 11738044
This is the form for adding the files

<html>

<head>
<title>Keyword profanity List</title>
</head>
<form name='settings' method='post' enctype="multipart/form-data" action='profanity_filter.php'>
<body bgcolor="white" text="black" link="blue" vlink="purple" alink="red">
            <p>Keyword profanity List</p>

                <p><input type="file" name="profanity" size="20"></p>

            <p>Keyword file</p>

                <p><input type="file" name="keywordlist"></p>

<p>Outputfilename</p>

    <p><input type="text" name="fileoutput"></p>

    <p><input type="submit" name="submit"></p>
</form>
</body>

</html>

<?php
$profanity = $_POST['profanity'];
    $keywordlist = $_POST['keywordlist'];
    $fileoutput = $_POST['fileoutput'];

//read lines of the bad word file into array
$badword = file($profanity);

//remove newline character from each array value
for ($i=0; $i<count($badword); $i++)
  $badword[$i] = rtrim($badword[$i]);

//read lines of the file to check
$lines = file($keywordlist);

$count = count($lines);

for ($i=0; $i<$count; $i++) {
  for ($j=0; $j<count($badword);$j++) {
    if ( false != stristr($lines[$i], $badword[$j]) ) {
      unset($lines[$i]);
    }
  }
}

$lines = array_values($lines);
$lines = implode("", $lines);

$handle = fopen("$fileoutput.txt", "w");
fwrite($handle, $lines);
fclose($handle);

?>

All i get is this

Warning: stristr(): Empty delimiter. in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 21

Warning: array_values(): The argument should be an array in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 27

Warning: implode(): Bad arguments. in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 28

Any ideas

0
 

Author Comment

by:playstat
ID: 11738409
its ok just finished it

thanks
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
Developers of all skill levels should learn to use current best practices when developing websites. However many developers, new and old, fall into the trap of using deprecated features because this is what so many tutorials and books tell them to u…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now