Solved

Servlet - xml response - Unconvertible UTF-8 character beginning with 0xbc

Posted on 2008-10-30
12
1,019 Views
Last Modified: 2013-11-24
I have this UTF encoding issue when parsing the response from flash. Flash generates  a xml file  to post some data some of it is in special characters.

On my servlet I have the following code. It breaks only if the respinse xml contains special characters giving UTF -8 error.

My Servlet is here.

      protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception {

            request.setCharacterEncoding("UTF-8");             
            Element user_data = new SAXBuilder().build(request.getInputStream()).getRootElement();
//// CODE BREAKS HERE GIVING UTF-8 ERROR.
            String flashVars = user_data.getChildText("account_number");
            System.out.println("flashVars is: " + flashVars);
            String account_number = flashVars.substring(0,flashVars.indexOf("|"));
            //String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.indexOf("|"));
            String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.length());

            System.out.println("?????????????????????????????????????????????????????????");
            System.out.println("account_number : " + account_number);
            System.out.println(user_data.getChildText("time_taken"));
            System.out.println("experiencePK : " + experiencePK);

Please advice. I found out that I  should put UTF-8 encoding to getBytes() method. Can any one show me how to do this.
.
0
Comment
Question by:istiaquem
  • 6
  • 4
  • 2
12 Comments
 
LVL 27

Expert Comment

by:BigRat
ID: 22842614
Have you actually checked the request content-type to be "text/xml" or "text/xml; charset=utf-8"? It looks like that the content-type is ANSI and not UTF-8.
0
 

Author Comment

by:istiaquem
ID: 22842701
Do I have to check this in my servlet? If yes can u please let me know how.  

I see the xml response from flash has the following

 contentType="text/xml"
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22842752
I have to go home now, but just as a test try setting the content-stream type to ANSI or ISO-8859-1.

(A problem is that mime type text/* is ISO-8859-1 whereas text/xml is UTF-8)
0
Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

 
LVL 86

Expert Comment

by:CEHJ
ID: 22849091
If you're sure it's UTF-8, the following is the safest strategy:

Element user_data = new SAXBuilder().build(new InpuStreamReader(request.getInputStream(), "UTF8")).getRootElement();
0
 

Author Comment

by:istiaquem
ID: 22851175
Dear friend.

I tried to implement that. I got this error:
2008-10-31 15:34:25,981 WARN  [org.springframework.web.servlet.DispatcherServlet] Handler execution resulted in exception - forwarding to resolved error view
org.jdom.input.JDOMParseException: Error on line 1: Illegal XML character:  .
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22851231
It seems then that the data from Flash is incorrect. Can you tell us (or post) exactly how you are getting the data?
0
 

Author Comment

by:istiaquem
ID: 22851359
This is the date i got from flash file.

 <?xml version="1.0" encoding="UTF-8"?><user_data><account_number>90|17</account_number><time_taken>????4????.???</time_taken><prize_id /></user_data>
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22851603
Yes, the data is corrupt, since you can't have hex 04 as a character in XML. But you also can't post stuff like that here. Can you either convert the text into hex or send it via e-mail to me (see my profile) and I'll have a look at it. It seems very strange that Flash has produced such a file. Have you checked that the clock on that machine is correct?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 22851944
Any binary data should be Base64 encoded and included in a CDATA section.
0
 
LVL 27

Accepted Solution

by:
BigRat earned 125 total points
ID: 22852072
istiaquem: The file must be sent as an attachment, otherwise I can't see what the content is. How did you get the data from the flash file?
0
 

Author Comment

by:istiaquem
ID: 22853008
Sure,  I will.

Here is how  I am able to get the data back from the xml.

InputStream in = request.getInputStream();

            String pay = null;
            try {
                  byte[] payload = new byte[contentLength];
                  int offset = 0;
                  int len = contentLength;
                  int byteCount;
                  while ( offset < contentLength ) {
                        byteCount = in.read( payload, offset, len );
                        if ( byteCount == -1 ) {
                              throw new ServletException( "Client did not send " + contentLength + " bytes as expected" );
                        }
                        offset += byteCount;
                        len -= byteCount;
                  }
                  pay = new String( payload, "UTF-8" );
                  System.out.println("new String( payload, ): " +pay );

            } finally {
                  if ( in != null ) {
                        in.close();
                  }
            }
servlet.java.txt
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22866114
I meant to say, please send the input file, which contains the bad characters, as an attachment by E-Mail.

Secondly I meant how did you get Flash to send you the file? From an HTML page with Javascript? or however?
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

The purpose of this article is to demonstrate how we can use conditional statements using Python.
Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question