Solved

Convert File to Base-10 number

Posted on 2002-07-05
53
396 Views
Last Modified: 2010-04-17
I would like to be able to open/read the bits of a file in Maple and convert them to a base-10 number.

For example I would like to know the command that will open a file called "file.txt". Then lets assume that the bits that consist this file are "0000000000010100". Then I would like this to get converted to a base-10 number. In this example to number: 2^4+2^2=20.

It would be extremely helpful if I could find a way to do that in Maple. However, if there is another utility that would do precisely that, this would be OK also.

The O/S that I am mostly interested is Win 98.
0
Comment
Question by:ody13
  • 23
  • 21
  • 5
  • +2
53 Comments
 

Expert Comment

by:danesbensen
ID: 7136896
Are you saying that the  file.txt is a BINARY file with some number of bits of binary data?  If so, is this always 32 bits...or?

0
 

Author Comment

by:ody13
ID: 7136914
The file.txt can be any file. It can be an excel document or it can be an image or it can be a mp3 song. Does this clarify the question or not really?

Also the number of bits on the file can be from a couple of hundred to ....

0
 

Expert Comment

by:danesbensen
ID: 7136961
"Also the number of bits on the file can be from a couple of hundred to ...."

So, lets say that the file has 200 bits in it.  What answer do you want to get back?  A single HUGE number that is 2^200 -- or a stream of 32 bit values...or?
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7137172
lets say you have a mp3 song with 16 million bits, which bits exactly (range, for example: 2'nd to 17'th) should be converted?
0
 

Author Comment

by:ody13
ID: 7137969
I would like to treat the stream of X million bits as one number and convert that number to a base-10 number.

Does this clarify what I am looking for or not really?

Thanks for trying to help me out.
0
 

Expert Comment

by:danesbensen
ID: 7138463
Lets say that there were just 60 bits in the file, that would be 2^60 which is in decimal -->  2305843009213693952

If there were 120 bits, then this would be a 36 digit base 10 number.  If there were just 500 bits, this would be am base 10 number over 150 digits long.  

Is this what you are looking for?  A HUGE base 10 number?  A number that, for a MP3 file, might easily be over 100,000 digits long?
0
 

Author Comment

by:ody13
ID: 7138599
Yes, this is correct. However, in an extreme example if a file had 500 bits and the first 499 (from the left to the right) were all 0 then the decimal number that would correspond to that number is 1.
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7138633
perl -e 'open(F,"yourfile");@a=stat(F);read(F,$b,$a[7]);print unpack("n*",$b);'
0
 
LVL 84

Expert Comment

by:ozo
ID: 7138903
perl -MMath::BigInt -0ne '$n=Math::BigInt->new(0);for(unpack"n*",$_){$n<<=16;$n+=$_}print$n;' yourfile
0
 

Author Comment

by:ody13
ID: 7139183
I will try the suggestions that you made, but are there any suggestions for a Windows environment?

Thanks
0
 
LVL 84

Expert Comment

by:ozo
ID: 7139245
In Windows, change the quotes:
perl -MMath::BigInt -0ne "$n=Math::BigInt->new(0);for(unpack'n*',$_){$n<<=16;$n+=$_}print$n;" yourfile
0
 

Author Comment

by:ody13
ID: 7139686
Where do I use this command, in MSDOS?

0
 
LVL 84

Expert Comment

by:ozo
ID: 7139691
in a dos shell
0
 

Author Comment

by:ody13
ID: 7139785
Ozo,

your code generates a lot of "+" signs on the number that outputs, whereas ahoffmann does not. Also for large files the two commands seem to generate different numbers, which one is the correct?

Finally, is there any way that I can have this output stored in a file and potentially be declared such that I can refer to this number from a C program or from a perl or ideally from Maple. If yes how exactly do I go about of doing that?

Thanks so much for your help!
0
 

Author Comment

by:ody13
ID: 7139830
ahoffmann and ozo,

in addition to the previous comment I created in notepad a text file that has the following characters:
"15 characters !"

Unless, I am screwing something up the decimal number for that should be:
255500069901247336988735086733238305

ahoffmann's command gives me:
1259782912672129281254602597029472

and ozo's:
+998047186889081975928098194092832


Any clues as to what is going on?


0
 
LVL 84

Expert Comment

by:ozo
ID: 7139856
Sorry, try:
perl -MMath::BigInt -0777e "open F,shift;binmode F;$n=Math::BigInt->new(0);for(unpack'C*',<F>){$n<<=8;$n+=$_}print$n;" yourfile
0
 
LVL 84

Expert Comment

by:ozo
ID: 7139902
I get
+255500079843604985837593137687765025
for
"15 characters !"
and
+255500069901247336988735086733238305
for
"15 CHARACTERS !"
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7140134
ody, I'm confused, a bit ...
in an earlier comment you said:
> .. example if a file had 500 bits and the first 499 (from the left to ..

so, I assume that you treat the whole file as exactly one number. You need to tell us how it should be interpreted: is it signed or not?
I'm not 100% shure about my suggestion, but if I look at ozo's last suggestion, I get the feeling that it does not use the whole file as a single number (ozo, please correct me if I'm wrong).
In fact, the binmode on open() is necessary on M$.

To write the result in a file, simply add following to the command (works on both: UNIX and M$'s command shell):

   > number.txt
0
 

Author Comment

by:ody13
ID: 7142160
ozo,

you were right I was using uppercase letters. Sorry for that.

I used the new command that you proposed and I have some comments:
1) It still generates the '+' signs (can we avoid them from being generated)

2) I used the command for a file that is 82.5KB and after 30 minutes there is no result generated. (why is this happening?)

3) Please see ahoffmann's comment from above and let me kow if he right (if he is what is the new command)

4) There seems to be a length of how big these perl commands are; is there any way that I can break it up?


ahoffmann,

Is there a way that I can refer to the number generated from a C program or from a perl or ideally from Maple. If yes how exactly do I go about of doing that?


Thanks!
Ody


0
 
LVL 84

Expert Comment

by:ozo
ID: 7142203
1) we can't avoid them being generated, but we can avoid printing them
2) The conversion routine is slow, especially on large files. (twice as big takes 4 times as much time)
3) is he right about what? with binmode and -0777, I should be reading the whole file.
4) You can put the perl commands in a file, in which case it would look like:
#!/usr/bin/perl
use Math::BigInt;
open F,shift;
binmode F;
$/=undef;
$n=Math::BigInt->new(0);
for( unpack 'C*', <F> ){
    $n<<=8;
    $n+=$_;
}
($_ = $n) =~ s/^\+//;
print;
0
 

Author Comment

by:ody13
ID: 7142412
Ozo,

could you please give me the exact command so that the "+" signs are not printed and that the output is printed on a file?


The conversion is much much slower than I anticipated. I would really appreciate if we would come up with a faster way?

Random Ideas:
Does the current logic skip the bits are 0's?
Can we create library's of different powers of 2 so that it does not go through the calculation for large powers, or it starts from a certain point and not from 2^1 everytime?

0
 
LVL 84

Expert Comment

by:ozo
ID: 7142424
#!/usr/bin/perl
use Math::BigInt;
my $inputfile = "yourfile";
my $outputfile = "number.txt";
open F,"<$inputfile" or die "can't open $inputfile because $!";
binmode F;
$/=undef;
my $n=Math::BigInt->new(0);
for( unpack 'C*', <F> ){
   $n<<=8;
   $n+=$_;
}
close F;
$_ = "$n";
s/^\+//;
open O,">$outputfile" or warn "can't open $inputfile because $!";
print O;
close O;
0
 
LVL 84

Expert Comment

by:ozo
ID: 7142430
Could you explain what you are trying to achieve by converting a large file into a single decimal number?
Perhaps we can find an easier way to accomplish the same goal?
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7142557
ozo,

> 3) is he right about what?
"I get the feeling that it does not use the whole file as a single number"
Means that I understand your code in that way that it reads byte by byte and append each to the previous one.
But now I see that it does not append but add. So the question is answered.

> .. converting a large file into a single decimal number?
is a huge, unique signature, somehow
0
 
LVL 24

Expert Comment

by:SunBow
ID: 7144812
>  I am mostly interested is Win 98
- (IMO,managing large decimals frequently suffers on matters of precision)
0
 

Author Comment

by:ody13
ID: 7147747
Ozo,

Basically, my purpose is to be able to convert a file (in the most efficient way) to a unique number so that I can apply all sorts of equations to this unique number.

Is the latest piece of code that you wrote any faster than the previous ones?


Ahoffmann,

I am not sure I am clear on your latest comments. Do you agree with Ozo that the whole file is read?




Thanks,
Ody
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 84

Expert Comment

by:ozo
ID: 7148083
convert a file (in the most efficient way) to a unique number

Then how about something like this?
#!/usr/bin/perl
my $inputfile = "yourfile";
my $outputfile = "number.txt";
open F,"<$inputfile" or die "can't open $inputfile because $!";
binmode F;
open O,">$outputfile" or warn "can't open $outfile because $!";
while( read F,$n,1 ){
    printf O "%03d",ord $n;
}
close O;
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7148637
> .. all sorts of equations to this unique number. ..
do you mean file1 > file2, file1 < file2, file1 == file2 ?
hmm, > and < can be done with stat(), == with a diff or cmp.
Why not simply build a hash/signature of the file?
0
 

Author Comment

by:ody13
ID: 7151738
Ahoffman,

I am not interested of just comparing the files (i.e. < or >). I am interested of being able to multiply, add, etc. to this unique number. Also from this unique number I would like to be able to recreate the file if needed.
0
 
LVL 84

Expert Comment

by:ozo
ID: 7151753
while( read F,$n,1 ){  printf O "%03d",ord $n;  }
Creates a unique number which you can multiply, add etc.
You can also recreate the file with
while( read F,$n,3 ){ printf "%c", $n }
0
 

Author Comment

by:ody13
ID: 7168001
Ozo,

one very elementary question:

What is the command I need to use to compile a file in perl?

I 'll try your latest suggestion ASAP and let you know how it worked.

Thanks,
Odiseas
0
 
LVL 84

Expert Comment

by:ozo
ID: 7168008
perl filename
0
 

Author Comment

by:ody13
ID: 7168122
Thanks Ozo,

I tried it and you are right it is very fast, much better than before!

I also tried your suggestion for recreating the original file and it worked fine with a text file, but when I tried to convert a jpg image to the unique number and then from the text file (that the unique number was printed) back to the jpg image it did not work, any clues? (the same happened with an mp3 song also)

BTW, the command that I used for that was:
while( read F,$n,3 ){ printf O "%c", $n }


Also since perl is so powerful could you please tell me how I can append to the code that you gave me some commands that would perform mathematical calculations (such as addition, multiplication, logarithms) to the output file (out1) and then output a second output file (out2) that is out1 after the calculations. I am guessing that it will be very slow performing calculations to the whole of out1 so I am open to the idea of braking it up to a number of pieces (the fewer the better)

Thanks so much!
0
 
LVL 84

Expert Comment

by:ozo
ID: 7168389
Did you set binmode on the output file when you tried a jpeg?
#!/usr/bin/perl
my $inputfile = "number.txt";
my $outputfile = "yourfile";
open F,"<$inputfile" or die "can't open $inputfile because $!";
open O,">$outputfile" or warn "can't open $outfile because $!";
binmode O;
while( read F,$n,3 ){
     printf O "%c", $n;
}
close O;
0
 
LVL 84

Expert Comment

by:ozo
ID: 7168396
To add two number files:
#!/usr/bin/perl
use Math::BigInt;
open F,"<out1.txt" or die $!;
$/=undef;
my $n=Math::BigInt->new(<F>);
open F,"<out2.txt" or die $!;
my $m=Math::BigInt->new(<F>);
$sum = $n+$m;
open O,">out2.txt" or warn $!;
print O $sum;
0
 

Author Comment

by:ody13
ID: 7168405
Setting binmode made it work.

My question was that I want to perform calculations on the number that is on out1.txt, like for example take the logarithm of that number and do other things as well and then get the output of these calculations on out2.txt.


Sorry for not being clear.
0
 

Author Comment

by:ody13
ID: 7253982
Hey Ozo,

you have been by far themost helpful in answering this question. As a last favor could you please take a stab on the above question?


Thanks so much,
Ody
0
 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
ID: 7254207
#!/usr/bin/perl
use Math::BigInt;
open O1,"<out1.txt" or die $!;
$/=undef;
my $x=Math::BigInt->new(<O1>);
close O1;
open O2,">out2.txt" or die $!;
print O2 log(join'',map{$x->digit(-$_)}(1..16)) +($x->length()-16)*log(10);
0
 

Author Comment

by:ody13
ID: 7254965
Ozo,

I am getting the following error:

Can't locate object method "digit" via package "Math::BigInt" (perhaps you forgot to load "Math::BigInt"?) at cal_conversion.txt line 8

Thanks,
Ody
0
 

Author Comment

by:ody13
ID: 7257720
Any clues regarding the error that I am getting?
0
 

Author Comment

by:ody13
ID: 7257721
Any clues regarding the error that I am getting?

Thanks so much for all your help,
Ody
0
 
LVL 84

Expert Comment

by:ozo
ID: 7257781
It sounds like you have an older version of Math::BigInt.
Try:
#!/usr/bin/perl
use Math::BigInt;
open O1,"<out1.txt" or die $!;
$/=undef;
my $x=Math::BigInt->new(<O1>);
close O1;
open O2,">out2.txt" or die $!;
($m,$e) = $" =~ /(\d{1,16})(.*)/;
print O2 (log $m)+(length $e)*log(10);
0
 
LVL 84

Expert Comment

by:ozo
ID: 7257783
($m,$e) = $x =~ /(\d{1,16})(.*)/;
0
 

Author Comment

by:ody13
ID: 7260476
Ozo,

I used the above script for a file that contained the following number: "100"

Since the script takes the log of that file shouldn't I had gotten "2" in the output file?

Instead I got "4.60517018598809". Anyhow, I am mostly interested to know what do I need to change to the script so that from that number I can get the original number (i.e. "100")

Thanks
0
 
LVL 84

Expert Comment

by:ozo
ID: 7260673
4.605170185988091 is log base e of 100
exp(4.605170185988091) will get back to 100
But this will only be approximate for larger numbers
0
 

Author Comment

by:ody13
ID: 7266021
Instead of taking the log is there another math calculation that you would recommend that would allow us to get back to the exact "initial" number?
0
 
LVL 84

Expert Comment

by:ozo
ID: 7266054
There are many possible reversible transforms, depending on what you want to do.
One simple transform could be to use tr///
0
 

Author Comment

by:ody13
ID: 7269401
What I want to do is convert the file out1.txt to a small number that will be printed in out2.txt in the most efficient/quick way. Then I want to be able to get back to out1.txt from out2.txt again in a fast way.

Do you mind writing the code with the transform that you believe is the most efficient?

Thanks for all your help.
0
 
LVL 84

Expert Comment

by:ozo
ID: 7269567
How small a number?  The number in out2.txt will have to be about the same size as the number in out1.txt
The most efficient/quick way is to make out1.txt and out2.txt identical.
Another simple transform might be to simply add 1 to the number in out1.txt to obtain out2.txt and subtract 1 to get back to out1.txt
I could write code to do these transforms if you wish,
but I'm not sure what goal you are trying to accomplish by doing this.
0
 

Author Comment

by:ody13
ID: 7269577
My goal is to have out2.txt be as small in size as possible, but also be a number that would allow me to get to precisely out1.txt

Thanks
0
 
LVL 84

Expert Comment

by:ozo
ID: 7270063
#!/usr/bin/perl
open A,"<out1.txt" or die "can't open out1.txt $!";
$/=undef;
$_ = <A>;
tr/0-9//dc;
open B,">out2.txt" or die "can't open out2.txt $!";
$x=substr($_,0,(length)%7||7,"");
print B substr pack("N",$x),-3;
while( $x = substr($_,0,7,"") ){
    print B substr pack("N",$x),-3;
}
close B;


#!/usr/bin/perl
open B,"<out2.txt" or die "can't open out2.txt $!";
$/=\3;
open A,">out1.txt" or die "can't open out1.txt $!";
print A unpack("N","\0".<B>);
while( <B> ){
    printf A "%07d",unpack("N","\0".$_);
    print A "\n";
}
close A;
0
 

Author Comment

by:ody13
ID: 7270474
This is fantastic! This is exactly what I wanted.

I would love to pick your mind for different transforms that the size of out2.txt can become even smaller. Should I post that as a seperate question? Do you think you will be able to help?

Thanks again
0
 
LVL 84

Expert Comment

by:ozo
ID: 7271289
I can get out2.txt down to 5/12 the size of out1.txt but to do much more than that would require knowing a lot more about the structure of out1.txt.  For example, are there long series of repeated zeros as in your initial example "0000000000010100"?
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
fix34  challenge 9 97
mergeTwo  challenge 13 72
File.WriteAllLines problem at random C# ASP.NET 6 53
topping1 challenge 7 51
I know it’s not a new topic to discuss and it has lots of online contents already available over the net. But Then I thought it would be useful to this site’s visitors and can have online repository on vim most commonly used commands. This post h…
If you’re thinking to yourself “That description sounds a lot like two people doing the work that one could accomplish,” you’re not alone.
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now