Script to Remove UniCode/Illegal Characters from Files' Name

Posted on 2007-12-05
Last Modified: 2010-04-21
Many files downloaded by eMule (ed2k/Kad) contain, in its name, UniCode characters (such as Chinese, Japanese, Korean, Arabic, Hebraic, Russian) which are seen as "Illegal Characters" by English version of Windows XP's explorer.exe... This causes serious troubles when managing such files...
Thus, I would like to get a script to automatically delete such characters from files' name, in order to avoid problems when trying to access them...

PS - even when we download an eBook totally written in English, stupidly the files' names contain such unicode/illegal characters...


Question by:asgarcymed
  • 3
  • 3

Expert Comment

ID: 20416989
use Renamer: (it supports unicode)

Author Comment

ID: 20420653
Although "ReNamer" is a superb renaming tool; I still to want/need to get a script, as a "light-weight" and quickly/rapidly solution "on-the-fly"...

Someone, outside EE, told me about REGULAR EXPRESSIONS...  Please give a look at:


My problem is that I do not know how to make the RegExp...

I can use VBScript, AutoIt, Perl, Python, Ruby, Tcl-Tk, or whatever, but I need some help...



Author Comment

ID: 20424849
I am now using "RegExBuddy", a superb Win32 app to work and learn about Regular Expressions...

Using Google, I could get a txt file (see which has many, many Chinese characters; and few English characters... I opened it with RegExBuddy, and I tested both RegEx's:




But the results of test/debug were very confusing...

Even more - I got the Windows XP MUI (MultiLingual User Interface) and I installed all languages I already announced (Chinese/Japanese/Korean/Arabic/Hebraic/Russian)...

My confusion is now even bigger - some apps can correctly load the Chinese characters (for example), but the majority of apps continue not to deal with such characters (they show "squares" or "???????????" or distorted characters like when we try to read a binary file with a text editor...

A big confusion is installed in my brain... Must I have MUI installed ?... What is the best RegEx to kill such characters from files' names? If I have MUI installed, do I need such regex/script?? What should I do to solve this question once and for all?

Is there any Chinese/Japanese/Korean/Arabic/Hebraic/Russia person here? If yes, how do you manage the characters' conflicts between your Native Language and English?

Help is very appreciated!

Thanks in advance.

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.


Accepted Solution

Wod earned 500 total points
ID: 20424924
You could try something like this in perl:

$string =~ s/^[\w\.]/_/g;

(replace all characters that are not word characters and not a period character (a-Z, 0-9 and "_" or ".") with "_")

$string would be your filename

Assisted Solution

Wod earned 500 total points
ID: 20424932
PS: Some applications don't support Unicode (even if the OS do)

Author Closing Comment

ID: 31413044
Thank you very much!!

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
how to pick files up from ftp via script 8 49
Check version 13 68
how to add fields in script 20 51
Automate switch config backup 3 45
cPanel is a Unix based web hosting control panel that provides a graphical interface and automation tools designed to simplify the process of hosting a web site. cPanel utilizes a 3 tier structure that provides functionality for administrators, rese…
Deploying a Microsoft Access application in a Citrix environment is not difficult but takes a few steps. However, Citrix system people are often of little help, as they typically know next to nothing about Access. The script provided here will take …
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now