Solved

Servlet - xml response - Unconvertible UTF-8 character beginning with 0xbc

Posted on 2008-10-30
12
997 Views
Last Modified: 2013-11-24
I have this UTF encoding issue when parsing the response from flash. Flash generates  a xml file  to post some data some of it is in special characters.

On my servlet I have the following code. It breaks only if the respinse xml contains special characters giving UTF -8 error.

My Servlet is here.

      protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception {

            request.setCharacterEncoding("UTF-8");             
            Element user_data = new SAXBuilder().build(request.getInputStream()).getRootElement();
//// CODE BREAKS HERE GIVING UTF-8 ERROR.
            String flashVars = user_data.getChildText("account_number");
            System.out.println("flashVars is: " + flashVars);
            String account_number = flashVars.substring(0,flashVars.indexOf("|"));
            //String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.indexOf("|"));
            String experiencePK = flashVars.substring(flashVars.indexOf("|") + 1, flashVars.length());

            System.out.println("?????????????????????????????????????????????????????????");
            System.out.println("account_number : " + account_number);
            System.out.println(user_data.getChildText("time_taken"));
            System.out.println("experiencePK : " + experiencePK);

Please advice. I found out that I  should put UTF-8 encoding to getBytes() method. Can any one show me how to do this.
.
0
Comment
Question by:istiaquem
  • 6
  • 4
  • 2
12 Comments
 
LVL 27

Expert Comment

by:BigRat
ID: 22842614
Have you actually checked the request content-type to be "text/xml" or "text/xml; charset=utf-8"? It looks like that the content-type is ANSI and not UTF-8.
0
 

Author Comment

by:istiaquem
ID: 22842701
Do I have to check this in my servlet? If yes can u please let me know how.  

I see the xml response from flash has the following

 contentType="text/xml"
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22842752
I have to go home now, but just as a test try setting the content-stream type to ANSI or ISO-8859-1.

(A problem is that mime type text/* is ISO-8859-1 whereas text/xml is UTF-8)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 22849091
If you're sure it's UTF-8, the following is the safest strategy:

Element user_data = new SAXBuilder().build(new InpuStreamReader(request.getInputStream(), "UTF8")).getRootElement();
0
 

Author Comment

by:istiaquem
ID: 22851175
Dear friend.

I tried to implement that. I got this error:
2008-10-31 15:34:25,981 WARN  [org.springframework.web.servlet.DispatcherServlet] Handler execution resulted in exception - forwarding to resolved error view
org.jdom.input.JDOMParseException: Error on line 1: Illegal XML character:  .
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22851231
It seems then that the data from Flash is incorrect. Can you tell us (or post) exactly how you are getting the data?
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 

Author Comment

by:istiaquem
ID: 22851359
This is the date i got from flash file.

 <?xml version="1.0" encoding="UTF-8"?><user_data><account_number>90|17</account_number><time_taken>????4????.???</time_taken><prize_id /></user_data>
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22851603
Yes, the data is corrupt, since you can't have hex 04 as a character in XML. But you also can't post stuff like that here. Can you either convert the text into hex or send it via e-mail to me (see my profile) and I'll have a look at it. It seems very strange that Flash has produced such a file. Have you checked that the clock on that machine is correct?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 22851944
Any binary data should be Base64 encoded and included in a CDATA section.
0
 
LVL 27

Accepted Solution

by:
BigRat earned 125 total points
ID: 22852072
istiaquem: The file must be sent as an attachment, otherwise I can't see what the content is. How did you get the data from the flash file?
0
 

Author Comment

by:istiaquem
ID: 22853008
Sure,  I will.

Here is how  I am able to get the data back from the xml.

InputStream in = request.getInputStream();

            String pay = null;
            try {
                  byte[] payload = new byte[contentLength];
                  int offset = 0;
                  int len = contentLength;
                  int byteCount;
                  while ( offset < contentLength ) {
                        byteCount = in.read( payload, offset, len );
                        if ( byteCount == -1 ) {
                              throw new ServletException( "Client did not send " + contentLength + " bytes as expected" );
                        }
                        offset += byteCount;
                        len -= byteCount;
                  }
                  pay = new String( payload, "UTF-8" );
                  System.out.println("new String( payload, ): " +pay );

            } finally {
                  if ( in != null ) {
                        in.close();
                  }
            }
servlet.java.txt
0
 
LVL 27

Expert Comment

by:BigRat
ID: 22866114
I meant to say, please send the input file, which contains the bad characters, as an attachment by E-Mail.

Secondly I meant how did you get Flash to send you the file? From an HTML page with Javascript? or however?
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now