• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1557
  • Last Modified:

UTF-8 String to Binary

Hi Experts,

I am actually getting a "application/octet-stream" response from a service, which works perfectly fine when I am streaming a XML or Text/Plain content, but when I am streaming a image is where the problem starts.

Looks like my client side (Adobe Flex) I am getting a string representation of this binary data, now all I want to do is to save it as the actual data itself.

Is there any way to do this? also please not by default Flex uses UTF-8 to store string data, hence the resulting file size as well as the encoding are wrong.

In simple terms all I want to know how to handle a Binary image which is now in the form of a UTF-8 String.

Also similar to that  of Java Flex lacks support for unsigned byte is that creating the problem?

0
Siva Prasanna Kumar
Asked:
Siva Prasanna Kumar
  • 10
  • 6
  • 4
  • +1
3 Solutions
 
CEHJCommented:
It might be an idea to post a file containing a known (small image) and also one containing the response to a request for that image
0
 
objectsCommented:
> how to handle a Binary image which is now in the form of a UTF-8 String.

not sure I totally follow but you can't read an image as a string unless it is actually be written as a string, for example after being Base64 Encoded.
If its binary data you need to read it as a binary stream, and not as a string.
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
Thanks for the quick response CHEJ, Ok here is the complete case, I am using MTOM attachment for transferring data files.

So here are the data files

I am attaching the actual File,

The file I wrote from FLEX client (which is the one needs to be fixed :) )

The one which I got from SOAP UI which is perfect (from the same service).

 
Test1.png
Test12.png
Test13.png
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
Hi Objects,

its a MTOM- XOP response, So its nor exactly a Base64Encoded Binary, I have previously dealt with Base64Encoded Binary via webservice that works perfectly too.

This is how the Raw Response looks:


HTTP/1.1 200 OK
Transfer-encoding: chunked
Content-type: multipart/related;start="<rootpart*bfaaf745-c158-4459-bdf5-65e87b993e61@example.jaxws.sun.com>";type="application/xop+xml";boundary="uuid:bfaaf745-c158-4459-bdf5-65e87b993e61";start-info="text/xml"

--uuid:bfaaf745-c158-4459-bdf5-65e87b993e61
Content-Id: <rootpart*bfaaf745-c158-4459-bdf5-65e87b993e61@example.jaxws.sun.com>
Content-Type: application/xop+xml;charset=utf-8;type="text/xml"
Content-Transfer-Encoding: binary

<?xml version="1.0" ?><S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><ns2:getFileResponse xmlns:ns2="soa2world.blogspot.com"><return><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:5c1336ee-2fd1-4c43-b03e-290af260e44e@example.jaxws.sun.com"></xop:Include></return></ns2:getFileResponse></S:Body></S:Envelope>
--uuid:bfaaf745-c158-4459-bdf5-65e87b993e61
Content-Id: <5c1336ee-2fd1-4c43-b03e-290af260e44e@example.jaxws.sun.com>
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
----------
‰PNG

Binary data, Direct paste didn't work!
-----
--uuid:bfaaf745-c158-4459-bdf5-65e87b993e61--

Open in new window

0
 
objectsCommented:
if its not base64 encode then its probably hex encoded
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
So you can almost assume that I have  the whole response which I pasted above as String, now I want to just extract the Binary attachment and save.
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
I don't think it encoded at all, thats the feature of XOP, its almost like a Mail attachment.
if you can see in my response
"‰PNG" which is the exact same string as you can view when you open PNG image in a text editor.
0
 
AnnOminousCommented:
You might want to confirm whether you actually have UTF-8 or a fixed length encoding.

Why don't you generate two images (of the same size) and see how the resulting output size differs? A constant color (white or black) and a random noise image would be good starting points.

UTF-8 should produce different sizes. Base64 should produce the same size.
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
@AnnOminous, If you see the attachment in my first comment, you can see the differences, its only the difference in the way i am (my Flex Client is writing it),

SOAP UI which is a standard soap testing tool, was able to write the exact file as show.

But my file has extra bytes, and wired chars :)

The service works perfectly fine for any text or XML file. Even My flex client works perfect for those files. But when I get a Binary attachment I am in no position to save as is. thats what I am trying to figure out!.

I have been trying various possible options from last two days, as I was running out of ways hence posted this question.

0
 
AnnOminousCommented:
One example does not distinguish between UTF8 and Base64 based on size alone.

If you try a text or XML file with diacritics like é, do you see it changed? For example, Montréal into Montr¿©al>?
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
Do you see my soap Request out there? it clearly shows that the data coming as part of the response is

The program works perfectly for Text/UTF-8 content.

I think we need to seeing on the difference b/w UTF-8 and ASCII (thats used for binary???)

Can some on help me how to convert a binary file which is now in UTF-8 format string needs to be converted back to Binary :)

Any way this is how it looks when I send a text file:

Attached are the actual response and the text file.
HTTP/1.1 200 OK
Transfer-encoding: chunked
Content-type: multipart/related;start="<rootpart*38650e78-6b12-4396-b94f-a97863269fc2@example.jaxws.sun.com>";type="application/xop+xml";boundary="uuid:38650e78-6b12-4396-b94f-a97863269fc2";start-info="text/xml"

--uuid:38650e78-6b12-4396-b94f-a97863269fc2
Content-Id: <rootpart*38650e78-6b12-4396-b94f-a97863269fc2@example.jaxws.sun.com>
Content-Type: application/xop+xml;charset=utf-8;type="text/xml"
Content-Transfer-Encoding: binary

<?xml version="1.0" ?><S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><ns2:getFileResponse xmlns:ns2="soa2world.blogspot.com"><return><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:5768dfd4-efee-4b39-91a3-6730f7c51e20@example.jaxws.sun.com"></xop:Include></return></ns2:getFileResponse></S:Body></S:Envelope>
--uuid:38650e78-6b12-4396-b94f-a97863269fc2
Content-Id: <5768dfd4-efee-4b39-91a3-6730f7c51e20@example.jaxws.sun.com>
Content-Type: text/plain
Content-Transfer-Encoding: binary

Test1243434
--uuid:38650e78-6b12-4396-b94f-a97863269fc2--

Open in new window

Test.txt
0
 
AnnOminousCommented:
Please send a text file with something other than 7-bit USASCII.

Send this text:
Les Européens s'accordent sur la Grèce.

Only the two diacritics should be garbled if it's UTF-8. As soon as you have non USASCII, you might get the whole thing encoded.
0
 
objectsCommented:
> I don't think it encoded at all

it has to be encoded.
What code are you using to read it?
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
sorry for the delay in responding, conclusion for this question is the fact that once the binary data is received as a UTF-8, there is almost no way to get back the data as binary, had to capture the data even before it gets converted to UTF-8 String , and did it successfully.

Shall I delete this question? because there is no real way to convert UTF-8 String to Binary at all.
0
 
objectsCommented:
I explained above that a string can only be converted to binary if it has been correctly encoded in the first place.
0
 
AnnOminousCommented:
UTF-8 is a non-lossy conversion and can encode binary without difficulty. Therefore, you can convert from UTF-8 back to binary without loss, just as you can convert from BASE64 to binary.

I'm not sure what the problem was that you where encountering, and you may have a way to avoid UTF8, but if you cant convert from UTF8 to binary, then you might not have had UTF8 at all.
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
@Objects exactly right the whole purpose of MTOM is to send the data directly as binary format just like a email attachment. if it was base64 encoded It might have been lot easier but it hinders the performance (size of the message is almost exponential when the attachment is huge)

@AnnOminous
"UTF-8 is a non-lossy conversion and can encode binary without difficulty." accpeted, but decoding??? the platform used need to have that support and adobe flex doesn't please don't think that UTF-8 is base64 encoded data, its a binary attachment, dude error in decoding is accessible as a UTF-8 String.
0
 
objectsCommented:
> @Objects exactly right the whole purpose of MTOM is to send the data directly as binary format just like a email attachment

if its binary content then you should be reading it as such, and not reading it as a UTF8 string (because it isn't one)
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
ya dude that's what ended up as my solution, I managed to trap it before it gets converted to UTF-8. that's why was thinking should I delete this question, as there is no real solution for it, if not I can promptly share the points.
0
 
CEHJCommented:
I'm not sure if there's a standard MTOM encoding - it could be implementation specific, in which case you'd need to get the correct implementation
0
 
CEHJCommented:
XOP i think is a standard, but that has both optimised and unoptimised versions. See http://wiki.ihe.net/index.php?title=XDS.b
0
 
objectsCommented:
> ya dude that's what ended up as my solution, I managed to trap it before it gets converted to UTF-8. that's why was thinking should I delete this question

which is what I originally suggested you do :)
0
 
Siva Prasanna KumarPrincipal Solutions ArchitectAuthor Commented:
Thanks for participating, and pointing out some good stuff :)
0

Featured Post

Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

  • 10
  • 6
  • 4
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now