Solved

Script to Remove UniCode/Illegal Characters from Files' Name

Posted on 2007-12-05
6
3,636 Views
Last Modified: 2010-04-21
Many files downloaded by eMule (ed2k/Kad) contain, in its name, UniCode characters (such as Chinese, Japanese, Korean, Arabic, Hebraic, Russian) which are seen as "Illegal Characters" by English version of Windows XP's explorer.exe... This causes serious troubles when managing such files...
Thus, I would like to get a script to automatically delete such characters from files' name, in order to avoid problems when trying to access them...

PS - even when we download an eBook totally written in English, stupidly the files' names contain such unicode/illegal characters...

Thanks.

Regards.
0
Comment
Question by:asgarcymed
  • 3
  • 3
6 Comments
 
LVL 7

Expert Comment

by:Wod
Comment Utility
use Renamer: http://www.den4b.com/ (it supports unicode)
0
 

Author Comment

by:asgarcymed
Comment Utility
Although "ReNamer" is a superb renaming tool; I still to want/need to get a script, as a "light-weight" and quickly/rapidly solution "on-the-fly"...

Someone, outside EE, told me about REGULAR EXPRESSIONS...  Please give a look at:

http://www.autoitscript.com/forum/index.php?showtopic=58848&st=0&gopid=444082&#entry444082

and

http://www.isthisthingon.org/unicode/allchars1.php

My problem is that I do not know how to make the RegExp...

I can use VBScript, AutoIt, Perl, Python, Ruby, Tcl-Tk, or whatever, but I need some help...


Thanks.

Regards.
0
 

Author Comment

by:asgarcymed
Comment Utility
I am now using "RegExBuddy", a superb Win32 app to work and learn about Regular Expressions...

Using Google, I could get a txt file (see http://www.xys.org/xys/netters/others/net/wiki2.txt) which has many, many Chinese characters; and few English characters... I opened it with RegExBuddy, and I tested both RegEx's:

[\x10-\x1F\x21-\x2F\x3A-\x40\x5B-\x60\x80-\xFF]

and

[^\u0000-\u024F]+


But the results of test/debug were very confusing...

Even more - I got the Windows XP MUI (MultiLingual User Interface) and I installed all languages I already announced (Chinese/Japanese/Korean/Arabic/Hebraic/Russian)...

My confusion is now even bigger - some apps can correctly load the Chinese characters (for example), but the majority of apps continue not to deal with such characters (they show "squares" or "???????????" or distorted characters like when we try to read a binary file with a text editor...

A big confusion is installed in my brain... Must I have MUI installed ?... What is the best RegEx to kill such characters from files' names? If I have MUI installed, do I need such regex/script?? What should I do to solve this question once and for all?

Is there any Chinese/Japanese/Korean/Arabic/Hebraic/Russia person here? If yes, how do you manage the characters' conflicts between your Native Language and English?

Help is very appreciated!

Thanks in advance.

Regards.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 7

Accepted Solution

by:
Wod earned 500 total points
Comment Utility
You could try something like this in perl:

$string =~ s/^[\w\.]/_/g;

(replace all characters that are not word characters and not a period character (a-Z, 0-9 and "_" or ".") with "_")

$string would be your filename
0
 
LVL 7

Assisted Solution

by:Wod
Wod earned 500 total points
Comment Utility
PS: Some applications don't support Unicode (even if the OS do)
0
 

Author Closing Comment

by:asgarcymed
Comment Utility
Thank you very much!!
Regards
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Over the years I have built up my own little library of code snippets that I refer to when programming or writing a script.  Many of these have come from the web or adaptations from snippets I find on the Web.  Periodically I add to them when I come…
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now