Solved

Hwo do I normalize utf8 characters in php

Posted on 2009-07-15
10
838 Views
Last Modified: 2012-05-07
I need to highlight search terms using php highlight_string but I run into issues when using wide character entries as in "foobar"   Note the full-width space in between.
when the original text to be highlighted contains only single space
"foo bar"  those words should be highlighted as well.

highlight_string("foobar", TRUE);

it seems to work the other way around highlight_string("foo bar") will properly highlight text that contains "foobar"

ideally no matter what user types(whatever space character - people can be cutting and pasting)  into search box I want that to be converted so the highlight_string works.

is there any utf8 character normalization function I can call?
0
Comment
Question by:zgecko
  • 5
  • 5
10 Comments
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24864310
Try running utf8_decode("foobar") before using highlight_string.
0
 

Author Comment

by:zgecko
ID: 24864655
That seems to convert my

"foobar" to  "foo?bar" not exactly what I want, but maybe I can work around "?"
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865318
If the text you're matching AGAINST is UTF-8, then you could try ENCODING instead of DECODING the seach string and then search. That's probably better than trying to convert UTF-8 back to plain ol' ASCII. Ultimately, you want your character encodings to match.

Also, most (if not all) of the str_ functions (like str_replace) are not multibyte-safe, so if highlight_string() makes use of str_replace or any str_ functions, try to use multibyte-safe functions:
http://us2.php.net/manual/en/book.mbstring.php
0
 

Author Comment

by:zgecko
ID: 24865363
Thank you I tried both encode and decode and I tried now the multibyte version
but I get following error
Fatal error: Call to undefined function mb_ereg_replace()
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865507
You'll need to enable the mbstring extension in PHP. In windows, it's just a matter of uncommenting the mbstring extension in the php.ini file. On Linux/etc operating systems, you may need to either recompile PHP or install the php-mbstring package (depending on how you installed PHP to begin with)
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 

Accepted Solution

by:
zgecko earned 0 total points
ID: 24865626
Thanks with that conversion and ? it seems to be working

print("<br />text<br />".$text);

$text=utf8_decode($text);

print("<br />text<br />".$text);

Open in new window

0
 

Author Comment

by:zgecko
ID: 24865640
Thank you for the help, unfortunately I can not just use mbstring extension, but I can ask why we do not use it
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865724
So the utf8_decode() solution does work (with a workaround) ?
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865728
Let me know if you have any further issues with it.
0
 

Author Comment

by:zgecko
ID: 24866681
not really it does convert to ? which might be  issue with other unicode characters I will need to test that, and inquire about possibility of turning on nultibyte support in php

0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
Developers of all skill levels should learn to use current best practices when developing websites. However many developers, new and old, fall into the trap of using deprecated features because this is what so many tutorials and books tell them to u…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now