Solved

Hwo do I normalize utf8 characters in php

Posted on 2009-07-15
10
850 Views
Last Modified: 2012-05-07
I need to highlight search terms using php highlight_string but I run into issues when using wide character entries as in "foobar"   Note the full-width space in between.
when the original text to be highlighted contains only single space
"foo bar"  those words should be highlighted as well.

highlight_string("foobar", TRUE);

it seems to work the other way around highlight_string("foo bar") will properly highlight text that contains "foobar"

ideally no matter what user types(whatever space character - people can be cutting and pasting)  into search box I want that to be converted so the highlight_string works.

is there any utf8 character normalization function I can call?
0
Comment
Question by:zgecko
  • 5
  • 5
10 Comments
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24864310
Try running utf8_decode("foobar") before using highlight_string.
0
 

Author Comment

by:zgecko
ID: 24864655
That seems to convert my

"foobar" to  "foo?bar" not exactly what I want, but maybe I can work around "?"
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865318
If the text you're matching AGAINST is UTF-8, then you could try ENCODING instead of DECODING the seach string and then search. That's probably better than trying to convert UTF-8 back to plain ol' ASCII. Ultimately, you want your character encodings to match.

Also, most (if not all) of the str_ functions (like str_replace) are not multibyte-safe, so if highlight_string() makes use of str_replace or any str_ functions, try to use multibyte-safe functions:
http://us2.php.net/manual/en/book.mbstring.php
0
 

Author Comment

by:zgecko
ID: 24865363
Thank you I tried both encode and decode and I tried now the multibyte version
but I get following error
Fatal error: Call to undefined function mb_ereg_replace()
0
 
LVL 34

Assisted Solution

by:gr8gonzo
gr8gonzo earned 500 total points
ID: 24865507
You'll need to enable the mbstring extension in PHP. In windows, it's just a matter of uncommenting the mbstring extension in the php.ini file. On Linux/etc operating systems, you may need to either recompile PHP or install the php-mbstring package (depending on how you installed PHP to begin with)
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 

Accepted Solution

by:
zgecko earned 0 total points
ID: 24865626
Thanks with that conversion and ? it seems to be working

print("<br />text<br />".$text);

$text=utf8_decode($text);

print("<br />text<br />".$text);

Open in new window

0
 

Author Comment

by:zgecko
ID: 24865640
Thank you for the help, unfortunately I can not just use mbstring extension, but I can ask why we do not use it
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865724
So the utf8_decode() solution does work (with a workaround) ?
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 24865728
Let me know if you have any further issues with it.
0
 

Author Comment

by:zgecko
ID: 24866681
not really it does convert to ? which might be  issue with other unicode characters I will need to test that, and inquire about possibility of turning on nultibyte support in php

0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
Part of the Global Positioning System A geocode (https://developers.google.com/maps/documentation/geocoding/) is the major subset of a GPS coordinate (http://en.wikipedia.org/wiki/Global_Positioning_System), the other parts being the altitude and t…
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to count occurrences of each item in an array.

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now