Want to know more about urlencode and urldecode

I know that urlencode function can convert some special characters and also Chinese characters into %xxx form.

However, what I'd like to know more about is the encoding mechanism of this function.

For example, how is it known that "%2C" stands for the "," character? And for Chinese characters, it would become even more complicated! For instance, the encoded word for the Chinese character "我" is "%A7%DA". How could this be done?!

Also, is there any mapping table for the convertion that I could refer to?

Many Thanks!!!
hellohelloworldAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

hernst42Commented:

Escape these bytes with the URI escaping mechanism (i.e., by converting each byte to %HH, where HH is the hexadecimal notation of the byte value).
See
http://www.w3.org/TR/html4/appendix/notes.html#h-B.2.2
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
hellohelloworldAuthor Commented:
Is there a way that I could find the mapping table in my PC?
0
hernst42Commented:
the mapping is easy to build

$table = array();
for ($i = 0; $i<= 255; ++$i) {
   $table[chr($i)] = url_encode(chr($i));
}

As said the character is just encoded as hex to get the value of a character use ord
see http://de3.php.net/manual/en/function.ord.php
0
Exploring ASP.NET Core: Fundamentals

Learn to build web apps and services, IoT apps, and mobile backends by covering the fundamentals of ASP.NET Core and  exploring the core foundations for app libraries.

hellohelloworldAuthor Commented:
yes, but what about the Chinese characters?
Is there anything I could trace so that if I see "%xyz%A23", I know that represents a certain Chinese character without using urldecode?
0
hernst42Commented:
No, you can't guess it by the format of the %xx%yy. As chines characters are stored als multi-byte-characters you might need a very long list (all chinese characters) to get those things known if %xx%yy is a chinese character.
0
hallvorsCommented:
What you want is possible but it is considerably more work that it is practical to put in. Just say decode and let PHP do the calculations :)

Anyway, thanks for an interesting question. Researching it taught me about both how UTF-8 works and about URL encoding in general.

First, link to an explanation of URL encoding:
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
(disclosure: it's written by someone I know ;)

Secondly, here is how to find the character from a URL encoding - manually!

Your character above - "&#25105;" (according to babelfish.altavista.com it means "I" in Chinese, if you can't see it in your browser try to copy this and paste in your address bar: javascript:'<html>&#25105;</html>' ) is actually encoded as %E6%88%91.

First tool we use is the Windows calculator: open it and change to Scientific mode in the View menu. Then choose "Hex" format and type the hex value from above (simply strip out the % -signs): e68891.

Now click the "Bin" option to get the binary value of this hexadecimal number. Copy it and paste it in Notepad.

111001101000100010010001

This is the binary, UTF-8 encoded string. We want to un-UTF-8 it to find the Unicode value. Here is a technical documentation for UTF-8:
ftp://ftp.isi.edu/in-notes/rfc2279.txt

First, start at the end of the string add linebreaks for each 8 digits.

11100110
10001000
10010001

From the first line, remove all the initial 1 - digits. From each of the next lines, remove the inital "10" - it will now look like this:

00110
001000
010001

Remove the line breaks and put it all on one line again:

00110001000010001

Copy that whole string and go back to the calculator. It should still be on "Binary" format, so just paste this new string.

If you now click "Dec" (for decimal or "normal" format), this is the exact number given in your first post because your browser translated a character not supported in the POST encoding to a HTML entity - 25105.

Next, click "Hex". The calculator will say "6211". Now open the Windows "character map" utility. Activate "Advanced view" if it doesn't show the "Go to Unicode" box. Then, in the "Go to Unicode" box type 6211. Voila, it shows the character you are looking for.

I'm sure you agree it is simpler to just type <? urldecode('%E6%88%91') ?> :-)
0
hallvordCommented:
I spent a long time on that reply though :(
and posted it with my wrong and now deleted profile :((
Oh well. It made me wiser and I also posted the mini-tutorial on my website..
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.