• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1356
  • Last Modified:

profanity filter from text file

Hi there

I have a list of words in a text file line by line and I need to use a text file that contains line by line profanity words. Then use that to find in another file text file and delete that line if it contains any of the words from the other file.

For example say the profanity word list is

air
bee
call

the text file with words is

air tree with leaves
bee fore you go there
I like trees
Not before dinner
every time after call
be have

then all that is left is

I like tree
not before dinner
be have

Can you make sure that the removed lines are collpased so they are after each other so there are no gaps in the output text file

thanks
 
0
playstat
Asked:
playstat
  • 3
  • 2
  • 2
1 Solution
 
newairaCommented:
try this:

<?php

//read lines of the bad word file into array
$badword = file('badwords.txt');

//remove newline character from each array value
for ($i=0; $i<count($badword); $i++)
  $badword[$i] = rtrim($badword[$i]);

//read lines of the file to check
$lines = file('listofwords.txt');

$count = count($lines);

for ($i=0; $i<$count; $i++) {
  for ($j=0; $j<count($badword);$j++) {
    if ( false != stristr($lines[$i], $badword[$j]) ) {
      unset($lines[$i]);
    }
  }
}

$lines = array_values($lines);
$lines = implode("", $lines);

$handle = fopen("newlistofwords.txt", "w");
fwrite($handle, $lines);
fclose($handle);

?>
0
 
snoyes_jwCommented:
Bear in mind that this method will also remove lines that contain bad words inside other words.  For example,  with your sample word list, you'll lose lines containing "Blair", "recall", "heebee-jeebies".  If that's a concern, you'll want to use regular expressions to check for word boundaries around the bad word.
0
 
newairaCommented:
Thanks for clarifying snoyes_jw.
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
playstatAuthor Commented:
could you add them please it would be most helpful

thanks
0
 
snoyes_jwCommented:
replace the line
if ( false != stristr($lines[$i], $badword[$j]) ) {
with these two:
$pattern = "/(?i)\b" . $badword[$j] . "\b/";
if (preg_match($pattern, $lines[i])) {

I think...
0
 
playstatAuthor Commented:
This is the form for adding the files

<html>

<head>
<title>Keyword profanity List</title>
</head>
<form name='settings' method='post' enctype="multipart/form-data" action='profanity_filter.php'>
<body bgcolor="white" text="black" link="blue" vlink="purple" alink="red">
            <p>Keyword profanity List</p>

                <p><input type="file" name="profanity" size="20"></p>

            <p>Keyword file</p>

                <p><input type="file" name="keywordlist"></p>

<p>Outputfilename</p>

    <p><input type="text" name="fileoutput"></p>

    <p><input type="submit" name="submit"></p>
</form>
</body>

</html>

<?php
$profanity = $_POST['profanity'];
    $keywordlist = $_POST['keywordlist'];
    $fileoutput = $_POST['fileoutput'];

//read lines of the bad word file into array
$badword = file($profanity);

//remove newline character from each array value
for ($i=0; $i<count($badword); $i++)
  $badword[$i] = rtrim($badword[$i]);

//read lines of the file to check
$lines = file($keywordlist);

$count = count($lines);

for ($i=0; $i<$count; $i++) {
  for ($j=0; $j<count($badword);$j++) {
    if ( false != stristr($lines[$i], $badword[$j]) ) {
      unset($lines[$i]);
    }
  }
}

$lines = array_values($lines);
$lines = implode("", $lines);

$handle = fopen("$fileoutput.txt", "w");
fwrite($handle, $lines);
fclose($handle);

?>

All i get is this

Warning: stristr(): Empty delimiter. in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 21

Warning: array_values(): The argument should be an array in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 27

Warning: implode(): Bad arguments. in C:\Program Files\nusphere\phped\Projects\profanity_filter.php on line 28

Any ideas

0
 
playstatAuthor Commented:
its ok just finished it

thanks
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

  • 3
  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now