Solved

How to block offensive / profane Spanish content

Posted on 2009-06-30
7
967 Views
Last Modified: 2013-12-09
Hi
We use Web Marshal and Mail Marshal to control our internet and email gateways respectively.

We now need to block Spanish offensive language from making its way to our users' inboxes, or blocking websites that contain offensive Spanish words. The problem is twofold:
a) Marshal provide no support in terms of custom text censor scripts that we can import
b) We cannot find any comprehensive csv list of offensive Spanish words that we can import into our Marshal suite, with the confidence that it will not produce false positives

Does someone know of a comprehensive csv list that we can utilize for instance?

Thanks
0
Comment
Question by:FphcareAdmins
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 28

Expert Comment

by:jhyiesla
ID: 24753243
If Spanish spam is like spam in English a list of offensive words will never work.  Let's say that the word "boob" is offensive.  So you block it. But then there is B**b, b0ob, and on and on. If your current filtering system depends on word lists to do it's job, you have the wrong system. I'm not familiar with the systems you are using so can't comment on them completely.  Over the years we have used such systems as SurfControl for web filtering and now use an appliance from St. Bernard. We also utilize the Postini email filtering system, but I honestly don't know if they deal with Spanish or not.
0
 

Expert Comment

by:Nemskinator
ID: 24759810
many spam applicances offer block filters specifically for languages such as spanish, chineese, russian and several other. it worked for me.
0
 

Expert Comment

by:Nemskinator
ID: 24759833
see attached
untitled.JPG
0
Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

 

Accepted Solution

by:
Gladinator earned 500 total points
ID: 24766942
You can swear until you are blue in the face but spammers have gone through several generations of spam since we used to block mails simply by specific words.

They can replace vowels with character symbols, and consonants with others (example: number 1 for letter L,  @ sign for letter O or zero).  The human brain can easily translate these into their proper equivalent and then become offended before a computer analysis will do the same.

The systems used by gateway spam products like you are using should already be more sophisticated than a straightforward "bad" word list.  Usually they have training features that administrators can use to educate the software "types of email we don't want to allow through our network".

Gather a large sample of offensive email (if your gateway system isnt already collecting it for analysis, collect someones home email address spam folder contents [in spanish of course]) and forward each mail to a mail admin account within the company, and use the grading/feedback system of the gateway to train it on what shouldn't get through.
0
 
LVL 6

Expert Comment

by:jwenting
ID: 24792470
and even if you block all those character substitutions, they'll just send a picture containing the text of the message, embedded in an html email.
Or an email containing a link to a website where the real message is to be found, masquerading in the email as something benign.

I've found that the most effective way to block spam is to figure out what TLDs most comes from and block those entirely.
This means blocking all of sub-Saharan Africa, most of South America (Chile seems clean, as does Peru), and parts of Asia (Indonesia, Burma, Vietnam, China, and a few others).
Blocking eastern Europe also helps.
You can always add whitelists to exclude specific addresses and domain names from the general block as needed.
0
 

Author Comment

by:FphcareAdmins
ID: 25901892
unfortunately we cannot block TLDs as we are a global company and receive emails etc from all over the globe
0
 

Author Closing Comment

by:FphcareAdmins
ID: 31598587
thankyou
0

Featured Post

Are You Headed to Black Hat USA 2017?

Getting ready for Black Hat next week? Kick things off with the WatchGuard Badge Challenge and test your puzzle and cipher skills. Do you have what it takes to earn our limited edition Firebox Badge? Get started today - https://crimsonthorn.net

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

UPDATE - 6/15/2011 Added support for Release Update 6 Maintenance Patch 2 Point Patch 1 (RU6 MP2 PP1). Fixed a defect in the username field that was hard-coded to look for a specific domain (left over code from testing). This release will be the …
An introduction to the wonderful sport of Scam Baiting.  Learn how to help fight scammers by beating them at their own game. This great pass time helps the world, while providing an endless source of entertainment. Enjoy!
Established in 1997, Technology Architects has become one of the most reputable technology solutions companies in the country. TA have been providing businesses with cost effective state-of-the-art solutions and unparalleled service that is designed…
Email security requires an ever evolving service that stays up to date with counter-evolving threats. The Email Laundry perform Research and Development to ensure their email security service evolves faster than cyber criminals. We apply our Threat…

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question