Solved

Hwo do I normalize utf8 characters in php

Posted on 2009-07-15
10
866 Views
Last Modified: 2012-05-07
I need to highlight search terms using php highlight_string but I run into issues when using wide character entries as in "foobar"   Note the full-width space in between.
when the original text to be highlighted contains only single space
"foo bar"  those words should be highlighted as well.

highlight_string("foobar", TRUE);

it seems to work the other way around highlight_string("foo bar") will properly highlight text that contains "foobar"

ideally no matter what user types(whatever space character - people can be cutting and pasting)  into search box I want that to be converted so the highlight_string works.

is there any utf8 character normalization function I can call?
0
Comment
Question by:zgecko
  • 5
  • 5
10 Comments
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24864310
Try running utf8_decode("foobar") before using highlight_string.
0
 

Author Comment

by:zgecko
ID: 24864655
That seems to convert my

"foobar" to  "foo?bar" not exactly what I want, but maybe I can work around "?"
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865318
If the text you're matching AGAINST is UTF-8, then you could try ENCODING instead of DECODING the seach string and then search. That's probably better than trying to convert UTF-8 back to plain ol' ASCII. Ultimately, you want your character encodings to match.

Also, most (if not all) of the str_ functions (like str_replace) are not multibyte-safe, so if highlight_string() makes use of str_replace or any str_ functions, try to use multibyte-safe functions:
http://us2.php.net/manual/en/book.mbstring.php
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 

Author Comment

by:zgecko
ID: 24865363
Thank you I tried both encode and decode and I tried now the multibyte version
but I get following error
Fatal error: Call to undefined function mb_ereg_replace()
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865507
You'll need to enable the mbstring extension in PHP. In windows, it's just a matter of uncommenting the mbstring extension in the php.ini file. On Linux/etc operating systems, you may need to either recompile PHP or install the php-mbstring package (depending on how you installed PHP to begin with)
0
 

Accepted Solution

by:
zgecko earned 0 total points
ID: 24865626
Thanks with that conversion and ? it seems to be working

print("<br />text<br />".$text);
$text=utf8_decode($text);
print("<br />text<br />".$text);

Open in new window

0
 

Author Comment

by:zgecko
ID: 24865640
Thank you for the help, unfortunately I can not just use mbstring extension, but I can ask why we do not use it
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865724
So the utf8_decode() solution does work (with a workaround) ?
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865728
Let me know if you have any further issues with it.
0
 

Author Comment

by:zgecko
ID: 24866681
not really it does convert to ? which might be  issue with other unicode characters I will need to test that, and inquire about possibility of turning on nultibyte support in php

0

Featured Post

Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will explain how to display the first page of your Microsoft Word documents (e.g. .doc, .docx, etc...) as images in a web page programatically. I have scoured the web on a way to do this unsuccessfully. The goal is to produce something …
I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

832 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question