[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2608
  • Last Modified:

Converting data in *.bin file to excel format

i have a file in *.bin format which contains numerical data from 0-255 (it is a remote sensing image).

bin file structure as follows:
header bytes: 0
pixels: 283
lines: 305
data type: unsigned character
bytes order:  LSB: Intel/VAX

the application i have allows me to view this image but ideally i need to work with this data in a spreadsheet (excel).  the application i'm using has no conversion/export utility so i will have do to this directly from the *.bin file.

any ideas on how to import directly from the *.bin file into excel?  any known utilities?

thanks in advance,
cwhelan
0
cwhelan33
Asked:
cwhelan33
  • 10
  • 5
  • 4
  • +3
1 Solution
 
ECNSSMTCommented:
Hi cwhelan33,

DMA related stuff is pretty obscure when it comes right down to it. As far as I know, if there is no inhouse conversion package, you may have to write it up yourself.  If you know the data format of the BIN file, you can get any 1st year CS major to write the binary to CSV text file in C or whatever language they are studying.  

So are you looking for oil... or rocks on Mars?

Regards,  
0
 
giltjrCommented:
What created the BIN file?

What application are you using to view it?  

What file types does the application allow you to save it as?
0
 
ECNSSMTCommented:
giltjr....,

chances are the data comes from some obscure mainframe app which may be housed on a VAX (???)

I kinda wondering if cwhelan33 needs this stuff displayed 283 x 305 cells.  If so, the next decision would be on how to trunc the stuff.  Excel goes up to 256 columns and 65536 rows.  (Of course there is no issues with the rows).

Also, if the data is to be manipulated, its usually on the bit level (ANDs, NANDs, ORs, NORs) and the like.   I haven't gone that route with Excel to determine if there is any easy way to do that.

And I think with LSB, you may have to switch the 2 byte word before the unsigned value is correct.  e.g. A9 vs 9A.  9A=154 decimal.

Regards  
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
giltjrCommented:
I am assuming that you think VAX because of the message:

     "bytes order:  LSB: Intel/VAX"

However. it also says Intel and both Intel and VAX use little endian (LSB to MSB) where as other platforms (IBM mainframe and some RISC based CPU's) use big endian (MSB to LSB).

My guess is the the message is a vauge attempt to say that the left most bit is the LSB (little endian).

0
 
cwhelan33Author Commented:
ESNSSMT,

thanks for the feedback and for the chuckle... i wish it was something as exciting as oil on mars!

the file contains classified landcover types and i'm trying to figure out the percentage of a specific landcover type (of a certain pixel value) within the image.  i'm using a model interface and i don't have access to the code.  frustrating this is that the interface allows me to view the pixel values on a cell by cell basis, but because of its limited functionality i can't export the values.

fyi - i'm working at a vertical & horizontal pixel resolution of 25 cells.

cwhelan
0
 
ECNSSMTCommented:
Hi cwhelan33,

LOL...  but I'm actually not kidding about oil... or rocks on Mars.  (Yes I have a dead pan way of talking too)  

The technology (or similiar technology) that you use is also used in oil exploration by oil companies and NASA for exploring plantary bodies.  If you are near (or know of) any college that has a geology department, they've probably face this dilemma also.  TALK TO YOUR SUPERVISORS BEFORE PROCEEDING, but if you can take a look that the file using a hex editor, corrolate that with the data you see via the application, you can get an associate that knows C or something to create an EASY program to translate that HEX FF to a text based numeric of 255; per the number of pixels and lines of data.  The difficult part is determining how the original programmers encoded everything.  

Again this field in the scheme of things is obscure.

@giltjr- yup that was what I was basing my assumption on.
0
 
scrathcyboyCommented:
remote sensed images cannot be converted to Excel files,  It doesnt make sense, Excel is a spreadsheet, and it doesnt show heavy graphic images (tho it can give links to them).  You need to convert this image to a standard TIF or other format, NASA uses TIFFs almost exclusively, surely you have way to convert image to TIFF format, otherwise, it is unusable.  All NASA people I know have unix algorithm to convert raw image data to TIFF.  No?
0
 
cwhelan33Author Commented:
scrathcyboy,

i'm not trying to view the image in excel.  i can already view the image in the interface i'm using.

i'm trying to view the all the image pixel 'values' at one time - the values that actual create the image.
0
 
ECNSSMTCommented:
remote sensing images are not pictures as regular people percieve them.  The images spoken of here would not be encoded in a TIFF, GIF, JPEG format.  

The numeric value here are interpreted in a different way; lets say, intensity associated with absorption.  

Don't worry about it not making sense, unless you are into geology it is very obscure.

so scrathcyboy you know people at NASA?  I don't...

Regards,
0
 
cwhelan33Author Commented:
ECNSSMT,

thanks for a providing a much more clear explanation than mine!  

i think i will have to pursue using some GIS software (ARCGis) or remote sensing software (PCI) to accomplish this.  i've used these in the past, but don't have a personal license for them.  

i was looking for a workaround/alternative way to do this, but alas, it may be easier to gain access to someone's lab afterall.

cwhelan
0
 
scrathcyboyCommented:
ecnssmt - I am into geology, and your explanation is no-sensical.  Explain.
0
 
ECNSSMTCommented:
Hi cwhelan33,

My pleasure.  You should have been around when this was done on the Apple IIe(s).

oddly (and sadly), if you go around to the colleges, I think this stuff is still at the "grassroots" level.

Regards,
0
 
scrathcyboyCommented:
pixel values on a cell by cell basis

This is the basis of any image translation, so what are you trying to do
0
 
scrathcyboyCommented:
think you are talking nonsense, there is no remote sensing image encoding that is not tranlatable to any standard image encoding.  Think you dont understand the latest andvances, the question and response indicate you are not aware of how to process the frame/scan data with the latest methods, so it seems.
0
 
ECNSSMTCommented:
Hi scrathcyboy,

I guess you didn't get too far in the geology degree, but there are certain ways of looking up at the moon, Mars and determine the composition of the celestrial body.  The planetary geologists at NASA should be able to explain it in detail, most of them by their 3rd or 4th year BA/BS degree and definitely while studying for their Masters have become very familiar with these techniques.  Unfortunately I don't have any contacts at NASA, so I can't refer you to any, but since you have some sort of association there, you may be better able to get better reference there.

Regards
0
 
cj_1969Commented:
A couple of things to address here ...
What app is creating the image file?  Is this a standard image file of some type ... yes, I read that it is a remote sensing image ... is this a standard image format for any application that does this type of thing or is this file proprietary to the application that is creating it?
If it is proprietary then first off, you should check with the vnedor of the application/image and see if you have the right to reverse engineer the content of the image.  Also, if this is proprietary, chances are you are not going to find anything out there that does what you are looking for, unless the vendor has written something already.

If you know that the numerical data is embeded in the file, then yes, it should be fairly easy to parse the file and pull out the values, this is a straight foward function, if you know the file/record/data format within the file .. or can figure it out.

If this is a true image file then you are talking about having to OCR the image to convert the graphical data to text.  If this is not a common graphical file format, then chances are you are out of luck, again, unless the vendor can provide a utility for doing this.

Have you checked with the provider of the image/data to see if they can provide just the numerical data that you want instead of trying to parse it out of the image?
0
 
ECNSSMTCommented:
Hi bruintje,

Can I make a motion to deduct points from scrathcyboy ??  during this session he's made at least 4 totally unsupportable and inaccurate points; 2 of them were in 1 sentence!!

Regards,
0
 
ECNSSMTCommented:
Hi bruintje,

I am not too sure if you are aware of the remote sensing field.  It is quite unique, as things goes.  There are not any mainstream products that can convert data readily from one of the number of formats that the genre creates.  If you want to get an understanding of the science behind it all and the software products that grew around it; you may want to google Remote Sensing as scrathcyboy did.  

The easiest way other than buying the products that costs in the six figure range is to code the program to actually do that; this takes a some understading of how the values are coded into a remote sensing image.  My best experience with this during my college years was to do that for a geologist friend of  mine.  (and it was a group effort at that).  

My advice to the user was  exactly to code the program to get the values he wanted to import into his Excel.  cwhelan33 instead decided to "borrow" time on an associate's program (which may be a undertaking itself) to do this in lieu of writing the progam.  

My issue with scrathcyboy is that he attempted to answer a question with ZERO background on this subject matter, STRETCHED his credibility and subsequently could have provided ERRONEOUS information.  Instead, he became a pest.  I can frankly say that he does not know remote sensing, is not a geologist (or has any background pertaining to it), has no affiliations with anyone in NASA (or even any secondary or tertiary contacts in terms of contractors or subcontractors).

So although cwhelan33 didn't utilize the answer I provided; it weighed in enough to cause him to seek a solution that was "readily" available to him.  This solution is only available to a "hand full of people" in a "hand full of companies";  ie. while Windows may be avaible to million of people; his solution is only available to 10s or 100s of people.  Please note that there may have be an authorization process involved.  see last comment made

"i was looking for a workaround/alternative way to do this, but alas, it may be easier to gain access to someone's lab afterall."

Regards,

0
 
cwhelan33Author Commented:
ECNSSMT & bruintje,

i did indeed gain access to a university lab that had remote sensing software (PCI) and was able to work with the file in question.

i agree that ECNSSMT provided the most helpful/knowledgable feedback for this question.  if is appropriate to award points based on this merit, rather than having the question "answered", i am happy to do so.  please inform me of the appropriate protocol for experts exchange in this manner.

cwhelan33
0
 
bruintjeCommented:
fair enough, i understand your point

i'm certainly not in aware of remote sensing, remote viewing maybe but not through software ;-)

the reason i thought this issue unresolved is that it didn't deliver a working software solution per se if you know what you're talking about [seems like it] then building software is a daunting task outside the scope of this question

BUT scratchyboy and you went a bit far in directly targetting each other in OT comments this is not done and i'm glad it didn't pan out into a flamewar which we rather don't store in the PAQ

you can always build a case on samples [links or code] that prove your point but not to disprove anothers point, that is left to the client

this is not EE policy but my interpretation of how the guidelines are being practiced :-)

to wrap it up since no is an answer at times

To the Moderator > PAQ the question with the points refunded to the client
0
 
ECNSSMTCommented:
thanks cwhelan33,
0
 
ECNSSMTCommented:
Hello bruintje,

I am unsure of how you interpreted my statement to scrathcyboy, but I just pointed out the mistatements he made.  I am unappreciative of the grandstanding and organizational name dropping that accompanied his statements.  And as for my statement; it was a single statement explaining the lineage towards understanding the subject matter at hand.  There was no intentions at starting a flame war.

Regards,
0
 
bruintjeCommented:
ah i was still typing [among other things] when cwhelan33 posted

to cwhelan33, you can always accept the most helpfull comment even if it didn't really solve the problem
0
 
cwhelan33Author Commented:
thanks bruintje,  i will award the points to ECNSSMT...

0

Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

  • 10
  • 5
  • 4
  • +3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now