?
Solved

Hwo do I normalize utf8 characters in php

Posted on 2009-07-15
10
Medium Priority
?
914 Views
Last Modified: 2012-05-07
I need to highlight search terms using php highlight_string but I run into issues when using wide character entries as in "foobar"   Note the full-width space in between.
when the original text to be highlighted contains only single space
"foo bar"  those words should be highlighted as well.

highlight_string("foobar", TRUE);

it seems to work the other way around highlight_string("foo bar") will properly highlight text that contains "foobar"

ideally no matter what user types(whatever space character - people can be cutting and pasting)  into search box I want that to be converted so the highlight_string works.

is there any utf8 character normalization function I can call?
0
Comment
Question by:zgecko
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
10 Comments
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 2000 total points
ID: 24864310
Try running utf8_decode("foobar") before using highlight_string.
0
 

Author Comment

by:zgecko
ID: 24864655
That seems to convert my

"foobar" to  "foo?bar" not exactly what I want, but maybe I can work around "?"
0
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 2000 total points
ID: 24865318
If the text you're matching AGAINST is UTF-8, then you could try ENCODING instead of DECODING the seach string and then search. That's probably better than trying to convert UTF-8 back to plain ol' ASCII. Ultimately, you want your character encodings to match.

Also, most (if not all) of the str_ functions (like str_replace) are not multibyte-safe, so if highlight_string() makes use of str_replace or any str_ functions, try to use multibyte-safe functions:
http://us2.php.net/manual/en/book.mbstring.php
0
7 Extremely Useful Linux Commands for Beginners

Just getting started with Linux? Here's a quick start guide that has 7 commands that we believe will come in handy.

 

Author Comment

by:zgecko
ID: 24865363
Thank you I tried both encode and decode and I tried now the multibyte version
but I get following error
Fatal error: Call to undefined function mb_ereg_replace()
0
 
LVL 35

Assisted Solution

by:gr8gonzo
gr8gonzo earned 2000 total points
ID: 24865507
You'll need to enable the mbstring extension in PHP. In windows, it's just a matter of uncommenting the mbstring extension in the php.ini file. On Linux/etc operating systems, you may need to either recompile PHP or install the php-mbstring package (depending on how you installed PHP to begin with)
0
 

Accepted Solution

by:
zgecko earned 0 total points
ID: 24865626
Thanks with that conversion and ? it seems to be working

print("<br />text<br />".$text);
$text=utf8_decode($text);
print("<br />text<br />".$text);

Open in new window

0
 

Author Comment

by:zgecko
ID: 24865640
Thank you for the help, unfortunately I can not just use mbstring extension, but I can ask why we do not use it
0
 
LVL 35

Expert Comment

by:gr8gonzo
ID: 24865724
So the utf8_decode() solution does work (with a workaround) ?
0
 
LVL 35

Expert Comment

by:gr8gonzo
ID: 24865728
Let me know if you have any further issues with it.
0
 

Author Comment

by:zgecko
ID: 24866681
not really it does convert to ? which might be  issue with other unicode characters I will need to test that, and inquire about possibility of turning on nultibyte support in php

0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Since pre-biblical times, humans have sought ways to keep secrets, and share the secrets selectively.  This article explores the ways PHP can be used to hide and encrypt information.
This article discusses how to implement server side field validation and display customized error messages to the client.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to count occurrences of each item in an array.
Suggested Courses

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question