Servlet - xml response - Unconvertible UTF-8 character beginning with 0xbc

I have this UTF encoding issue when parsing the response from flash. Flash generates  a xml file  to post some data some of it is in special characters.

On my servlet I have the following code. It breaks only if the respinse xml contains special characters giving UTF -8 error.

My Servlet is here.

      protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception {

            request.setCharacterEncoding("UTF-8");             
            Element user_data = new SAXBuilder().build(request.getInputStream()).getRootElement();
//// CODE BREAKS HERE GIVING UTF-8 ERROR.
            String flashVars = user_data.getChildText("account_number");
            System.out.println("flashVars is: " + flashVars);
            String account_number = flashVars.substring(0,flashVars.indexOf("|"));
            //String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.indexOf("|"));
            String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.length());

            System.out.println("?????????????????????????????????????????????????????????");
            System.out.println("account_number : " + account_number);
            System.out.println(user_data.getChildText("time_taken"));
            System.out.println("experiencePK : " + experiencePK);

Please advice. I found out that I  should put UTF-8 encoding to getBytes() method. Can any one show me how to do this.
.
istiaquemAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

BigRatCommented:
Have you actually checked the request content-type to be "text/xml" or "text/xml; charset=utf-8"? It looks like that the content-type is ANSI and not UTF-8.
0
istiaquemAuthor Commented:
Do I have to check this in my servlet? If yes can u please let me know how.  

I see the xml response from flash has the following

 contentType="text/xml"
0
BigRatCommented:
I have to go home now, but just as a test try setting the content-stream type to ANSI or ISO-8859-1.

(A problem is that mime type text/* is ISO-8859-1 whereas text/xml is UTF-8)
0
Cloud Class® Course: Certified Penetration Testing

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

CEHJCommented:
If you're sure it's UTF-8, the following is the safest strategy:

Element user_data = new SAXBuilder().build(new InpuStreamReader(request.getInputStream(), "UTF8")).getRootElement();
0
istiaquemAuthor Commented:
Dear friend.

I tried to implement that. I got this error:
2008-10-31 15:34:25,981 WARN  [org.springframework.web.servlet.DispatcherServlet] Handler execution resulted in exception - forwarding to resolved error view
org.jdom.input.JDOMParseException: Error on line 1: Illegal XML character:  .
0
BigRatCommented:
It seems then that the data from Flash is incorrect. Can you tell us (or post) exactly how you are getting the data?
0
istiaquemAuthor Commented:
This is the date i got from flash file.

 <?xml version="1.0" encoding="UTF-8"?><user_data><account_number>90|17</account_number><time_taken>????4????.???</time_taken><prize_id /></user_data>
0
BigRatCommented:
Yes, the data is corrupt, since you can't have hex 04 as a character in XML. But you also can't post stuff like that here. Can you either convert the text into hex or send it via e-mail to me (see my profile) and I'll have a look at it. It seems very strange that Flash has produced such a file. Have you checked that the clock on that machine is correct?
0
CEHJCommented:
Any binary data should be Base64 encoded and included in a CDATA section.
0
BigRatCommented:
istiaquem: The file must be sent as an attachment, otherwise I can't see what the content is. How did you get the data from the flash file?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
istiaquemAuthor Commented:
Sure,  I will.

Here is how  I am able to get the data back from the xml.

InputStream in = request.getInputStream();

            String pay = null;
            try {
                  byte[] payload = new byte[contentLength];
                  int offset = 0;
                  int len = contentLength;
                  int byteCount;
                  while ( offset < contentLength ) {
                        byteCount = in.read( payload, offset, len );
                        if ( byteCount == -1 ) {
                              throw new ServletException( "Client did not send " + contentLength + " bytes as expected" );
                        }
                        offset += byteCount;
                        len -= byteCount;
                  }
                  pay = new String( payload, "UTF-8" );
                  System.out.println("new String( payload, ): " +pay );

            } finally {
                  if ( in != null ) {
                        in.close();
                  }
            }
servlet.java.txt
0
BigRatCommented:
I meant to say, please send the input file, which contains the bad characters, as an attachment by E-Mail.

Secondly I meant how did you get Flash to send you the file? From an HTML page with Javascript? or however?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java EE

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.