How to Convert a File From ANSI to UTF8

jeff Lee
jeff Lee used Ask the Experts™
on
I save text files created in some kind of program developed by delphi 5.The default type of text encoding is ANSI. I don't want to use notepad to convert a  File From ANSI to UTF8.

So how can I convert a  File From ANSI to UTF8 by delphi 5? Thanks.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
ste5anSenior Developer

Commented:
Delphi 5? Really?

In actual Version you would use System.AnsiToUtf8. But I really can't remember whether this was already available in 1999.

When was WideString introduced to Delphi? Here you could use Utf8Encode.

Otherwise use the Win32 WideCharToMultiByte function.

Author

Commented:
Hi ste5an,

  I can't use the Utf8Encode function in Delphi 5.So would you please to explain WideCharToMultiByte function.For example,how to call the WideCharToMultiByte function in Delphi 5?

  I really appreciate for your help.
ste5anSenior Developer

Commented:
I only have a D7, so I cannot test it. And I really don't remember, whether WideString/AnsiString where already implemented.

Maybe it's something like

function StringToWideStringEx(const S: String; CodePage: Word): WideString;
var
    L: Integer;
begin
    L:= MultiByteToWideChar(CodePage, 0, PChar(S), -1, nil, 0);
    SetLength(Result, L-1);
    MultiByteToWideChar(CodePage, 0, PChar(S), -1, PWideChar(Result), L - 1);
end;

function WideStringToStringEx(const WS: WideString; CodePage: Word): String;
var
    L: Integer;
begin
    L := WideCharToMultiByte(CodePage, 0, PWideChar(WS), -1, nil, 0, nil, nil);
    SetLength(Result, L-1);
    WideCharToMultiByte(CodePage, 0, PWideChar(WS), -1, PChar(Result), L - 1, nil, nil);
end;

Open in new window

Just for curiosity: Why did you never upgrade?

Author

Commented:
Hi ste5an,

  I beg your pardon for applying late.

  The following is my code.
 
  var F : System.Text;
      findstr:string;

  savedialog1.filename :='12345678.'+'.U8';
  AssignFile(F,savedialog1.filename);
  Rewrite(F);

  findstr:='123456789';
  .......

  writeln(F, WideStringToStringEx(findstr,950));

  When I save the file, the type of file name is still ANSI. Can yo give me a hint? Thanks.

  Why never upgrade? ya,it bothered me a long time.I bought Delphi XE5,but I
can't find Third-party software component to develop my program.I fact,I must use a large number of Third-party software component . That's the main reason.
Can you give me some suggestions?
Sinisa VukSoftware architect
Top Expert 2012

Commented:
You need to write BOM bytes at first beginning of the file:
 Write(F, #$FEFF);

Open in new window


...those bytes tell that text file is unicode (utf-8) encoded.

Author

Commented:
Hi Sinisa Vuk,

  Thank you for your reply. I tune my code as follows. But it still can't work.The error message is Illegal type in Write/Written statement. Is there something wrong?

 savedialog1.filename :='12345678.'+'.U8';
  AssignFile(F,savedialog1.filename);
  Write(F, #$FEFF);   <---------add the line
  Rewrite(F);

  findstr:='123456789';
  .......

  writeln(F, WideStringToStringEx(findstr,950));

  regards,

 jeffI
Software architect
Top Expert 2012
Commented:
Sorry, I do not use AssignFile (direct io functions) because think that this is not good approach. Better use Openfile, WriteFile,...
But ... here is full example:
var
  F: TextFile;
  fileName: string;
begin
  fileName := ExtractFilePath(ParamStr(0)) + 'test.txt';
  AssignFile(F,fileName);
  Rewrite(F);
  Write(F,#$EF#$BB#$BF);
  Writeln(F,UTF8Encode('Text to test...'));
  CloseFile(F);

Open in new window

Author

Commented:
Hi Sinisa Vuk,

   I think your answer is very close to the real solution. But I can't use the UTF8Encode function in Delphi 5. I am looking for the UTF8Encode function. Thanks.


   regards,

jeff
Sinisa VukSoftware architect
Top Expert 2012
Commented:
For that part - you can use jcl components (JclUnicode) or Unicode lib by Mike Lischke (preferred - no need to install).

Author

Commented:
Hi,

  I found a tool to convert to utf-8 as follows:
But it doesn't seem to work.Such as Writeln(F,AnsiToUTF8('Text to test...'));   Could anyone help me? Thanks.

unit util_utf8;

interface
 
uses Windows;
 
type
  UTF8String = AnsiString;
 
  function AnsiToWide(const S: AnsiString): WideString;
  function WideToUTF8(const WS: WideString): UTF8String;
  function AnsiToUTF8(const S: AnsiString): UTF8String;
  function UTF8ToWide(const US: UTF8String): WideString;
  function WideToAnsi(const WS: WideString): AnsiString;
  function UTF8ToAnsi(const S: UTF8String): AnsiString;
 
implementation
 
function AnsiToWide(const S: AnsiString): WideString;
var len: integer;
    ws: WideString;
begin
Result:='';
if (Length(S) = 0) then
  exit;
len:=MultiByteToWideChar(CP_ACP, 0, PChar(s), -1, nil, 0);
SetLength(ws, len);
MultiByteToWideChar(CP_ACP, 0, PChar(s), -1, PWideChar(ws), len);
Result:=ws;
end;
 
function WideToUTF8(const WS: WideString): UTF8String;
var len: integer;
    us: UTF8String;
begin
Result:='';
if (Length(WS) = 0) then
  exit;
len:=WideCharToMultiByte(CP_UTF8, 0, PWideChar(WS), -1, nil, 0, nil, nil);
SetLength(us, len);
WideCharToMultiByte(CP_UTF8, 0, PWideChar(WS), -1, PChar(us), len, nil, nil);
Result:=us;
end;
 
function AnsiToUTF8(const S: AnsiString): UTF8String;
begin
Result:=WideToUTF8(AnsiToWide(S));
end;
 
function UTF8ToWide(const US: UTF8String): WideString;
var len: integer;
    ws: WideString;
begin
Result:='';
if (Length(US) = 0) then
  exit;
len:=MultiByteToWideChar(CP_UTF8, 0, PChar(US), -1, nil, 0);
SetLength(ws, len);
MultiByteToWideChar(CP_UTF8, 0, PChar(US), -1, PWideChar(ws), len);
Result:=ws;
end;
 
function WideToAnsi(const WS: WideString): AnsiString;
var len: integer;
    s: AnsiString;
begin
Result:='';
if (Length(WS) = 0) then
  exit;
len:=WideCharToMultiByte(CP_ACP, 0, PWideChar(WS), -1, nil, 0, nil, nil);
SetLength(s, len);
WideCharToMultiByte(CP_ACP, 0, PWideChar(WS), -1, PChar(s), len, nil, nil);
Result:=s;
end;
 
function UTF8ToAnsi(const S: UTF8String): AnsiString;
begin
Result:=WideToAnsi(UTF8ToWide(S));
end;
 
end.

Author

Commented:
Hi Sinisa Vuk ,

   Your suggestion is very helpful for me. Thanks again.
ste5anSenior Developer

Commented:
Why never upgrade? ya,it bothered me a long time.I bought Delphi XE5,but I
can't find Third-party software component to develop my program.I fact,I must use a large number of Third-party software component .
Well, you should look at JEDI (free) and DevExpress (commercial) for replacements.

Depending on the number of libraries and the structure of your code, you should really invest this time, cause code quality often gets better by a rewrite. Furthermore, depending on the kind of those libraries, functionality may break sooner or later due to Win10 incompatibilities.

Components which are so old, that they are no longer supported, are a technical debt.

New Delphi versions have new features (language, IDE). Those features allow faster coding, testing and deployment.

Author

Commented:
Hi ste5an,

  Your suggestion are precious to me. I really want to update the delphi versions a long time ago, Now you give me a direction. Thanks.

  best regards,

jeff

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial