Solved

Hwo do I normalize utf8 characters in php

Posted on 2009-07-15
10
882 Views
Last Modified: 2012-05-07
I need to highlight search terms using php highlight_string but I run into issues when using wide character entries as in "foobar"   Note the full-width space in between.
when the original text to be highlighted contains only single space
"foo bar"  those words should be highlighted as well.

highlight_string("foobar", TRUE);

it seems to work the other way around highlight_string("foo bar") will properly highlight text that contains "foobar"

ideally no matter what user types(whatever space character - people can be cutting and pasting)  into search box I want that to be converted so the highlight_string works.

is there any utf8 character normalization function I can call?
0
Comment
Question by:zgecko
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
10 Comments
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24864310
Try running utf8_decode("foobar") before using highlight_string.
0
 

Author Comment

by:zgecko
ID: 24864655
That seems to convert my

"foobar" to  "foo?bar" not exactly what I want, but maybe I can work around "?"
0
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865318
If the text you're matching AGAINST is UTF-8, then you could try ENCODING instead of DECODING the seach string and then search. That's probably better than trying to convert UTF-8 back to plain ol' ASCII. Ultimately, you want your character encodings to match.

Also, most (if not all) of the str_ functions (like str_replace) are not multibyte-safe, so if highlight_string() makes use of str_replace or any str_ functions, try to use multibyte-safe functions:
http://us2.php.net/manual/en/book.mbstring.php
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:zgecko
ID: 24865363
Thank you I tried both encode and decode and I tried now the multibyte version
but I get following error
Fatal error: Call to undefined function mb_ereg_replace()
0
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865507
You'll need to enable the mbstring extension in PHP. In windows, it's just a matter of uncommenting the mbstring extension in the php.ini file. On Linux/etc operating systems, you may need to either recompile PHP or install the php-mbstring package (depending on how you installed PHP to begin with)
0
 

Accepted Solution

by:
zgecko earned 0 total points
ID: 24865626
Thanks with that conversion and ? it seems to be working

print("<br />text<br />".$text);
$text=utf8_decode($text);
print("<br />text<br />".$text);

Open in new window

0
 

Author Comment

by:zgecko
ID: 24865640
Thank you for the help, unfortunately I can not just use mbstring extension, but I can ask why we do not use it
0
 
LVL 35

Expert Comment

by:gr8gonzo
ID: 24865724
So the utf8_decode() solution does work (with a workaround) ?
0
 
LVL 35

Expert Comment

by:gr8gonzo
ID: 24865728
Let me know if you have any further issues with it.
0
 

Author Comment

by:zgecko
ID: 24866681
not really it does convert to ? which might be  issue with other unicode characters I will need to test that, and inquire about possibility of turning on nultibyte support in php

0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Developers of all skill levels should learn to use current best practices when developing websites. However many developers, new and old, fall into the trap of using deprecated features because this is what so many tutorials and books tell them to u…
Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question