Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 433
  • Last Modified:

Convert File to Base-10 number

I would like to be able to open/read the bits of a file in Maple and convert them to a base-10 number.

For example I would like to know the command that will open a file called "file.txt". Then lets assume that the bits that consist this file are "0000000000010100". Then I would like this to get converted to a base-10 number. In this example to number: 2^4+2^2=20.

It would be extremely helpful if I could find a way to do that in Maple. However, if there is another utility that would do precisely that, this would be OK also.

The O/S that I am mostly interested is Win 98.
0
ody13
Asked:
ody13
  • 23
  • 21
  • 5
  • +2
1 Solution
 
danesbensenCommented:
Are you saying that the  file.txt is a BINARY file with some number of bits of binary data?  If so, is this always 32 bits...or?

0
 
ody13Author Commented:
The file.txt can be any file. It can be an excel document or it can be an image or it can be a mp3 song. Does this clarify the question or not really?

Also the number of bits on the file can be from a couple of hundred to ....

0
 
danesbensenCommented:
"Also the number of bits on the file can be from a couple of hundred to ...."

So, lets say that the file has 200 bits in it.  What answer do you want to get back?  A single HUGE number that is 2^200 -- or a stream of 32 bit values...or?
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
ahoffmannCommented:
lets say you have a mp3 song with 16 million bits, which bits exactly (range, for example: 2'nd to 17'th) should be converted?
0
 
ody13Author Commented:
I would like to treat the stream of X million bits as one number and convert that number to a base-10 number.

Does this clarify what I am looking for or not really?

Thanks for trying to help me out.
0
 
danesbensenCommented:
Lets say that there were just 60 bits in the file, that would be 2^60 which is in decimal -->  2305843009213693952

If there were 120 bits, then this would be a 36 digit base 10 number.  If there were just 500 bits, this would be am base 10 number over 150 digits long.  

Is this what you are looking for?  A HUGE base 10 number?  A number that, for a MP3 file, might easily be over 100,000 digits long?
0
 
ody13Author Commented:
Yes, this is correct. However, in an extreme example if a file had 500 bits and the first 499 (from the left to the right) were all 0 then the decimal number that would correspond to that number is 1.
0
 
ahoffmannCommented:
perl -e 'open(F,"yourfile");@a=stat(F);read(F,$b,$a[7]);print unpack("n*",$b);'
0
 
ozoCommented:
perl -MMath::BigInt -0ne '$n=Math::BigInt->new(0);for(unpack"n*",$_){$n<<=16;$n+=$_}print$n;' yourfile
0
 
ody13Author Commented:
I will try the suggestions that you made, but are there any suggestions for a Windows environment?

Thanks
0
 
ozoCommented:
In Windows, change the quotes:
perl -MMath::BigInt -0ne "$n=Math::BigInt->new(0);for(unpack'n*',$_){$n<<=16;$n+=$_}print$n;" yourfile
0
 
ody13Author Commented:
Where do I use this command, in MSDOS?

0
 
ozoCommented:
in a dos shell
0
 
ody13Author Commented:
Ozo,

your code generates a lot of "+" signs on the number that outputs, whereas ahoffmann does not. Also for large files the two commands seem to generate different numbers, which one is the correct?

Finally, is there any way that I can have this output stored in a file and potentially be declared such that I can refer to this number from a C program or from a perl or ideally from Maple. If yes how exactly do I go about of doing that?

Thanks so much for your help!
0
 
ody13Author Commented:
ahoffmann and ozo,

in addition to the previous comment I created in notepad a text file that has the following characters:
"15 characters !"

Unless, I am screwing something up the decimal number for that should be:
255500069901247336988735086733238305

ahoffmann's command gives me:
1259782912672129281254602597029472

and ozo's:
+998047186889081975928098194092832


Any clues as to what is going on?


0
 
ozoCommented:
Sorry, try:
perl -MMath::BigInt -0777e "open F,shift;binmode F;$n=Math::BigInt->new(0);for(unpack'C*',<F>){$n<<=8;$n+=$_}print$n;" yourfile
0
 
ozoCommented:
I get
+255500079843604985837593137687765025
for
"15 characters !"
and
+255500069901247336988735086733238305
for
"15 CHARACTERS !"
0
 
ahoffmannCommented:
ody, I'm confused, a bit ...
in an earlier comment you said:
> .. example if a file had 500 bits and the first 499 (from the left to ..

so, I assume that you treat the whole file as exactly one number. You need to tell us how it should be interpreted: is it signed or not?
I'm not 100% shure about my suggestion, but if I look at ozo's last suggestion, I get the feeling that it does not use the whole file as a single number (ozo, please correct me if I'm wrong).
In fact, the binmode on open() is necessary on M$.

To write the result in a file, simply add following to the command (works on both: UNIX and M$'s command shell):

   > number.txt
0
 
ody13Author Commented:
ozo,

you were right I was using uppercase letters. Sorry for that.

I used the new command that you proposed and I have some comments:
1) It still generates the '+' signs (can we avoid them from being generated)

2) I used the command for a file that is 82.5KB and after 30 minutes there is no result generated. (why is this happening?)

3) Please see ahoffmann's comment from above and let me kow if he right (if he is what is the new command)

4) There seems to be a length of how big these perl commands are; is there any way that I can break it up?


ahoffmann,

Is there a way that I can refer to the number generated from a C program or from a perl or ideally from Maple. If yes how exactly do I go about of doing that?


Thanks!
Ody


0
 
ozoCommented:
1) we can't avoid them being generated, but we can avoid printing them
2) The conversion routine is slow, especially on large files. (twice as big takes 4 times as much time)
3) is he right about what? with binmode and -0777, I should be reading the whole file.
4) You can put the perl commands in a file, in which case it would look like:
#!/usr/bin/perl
use Math::BigInt;
open F,shift;
binmode F;
$/=undef;
$n=Math::BigInt->new(0);
for( unpack 'C*', <F> ){
    $n<<=8;
    $n+=$_;
}
($_ = $n) =~ s/^\+//;
print;
0
 
ody13Author Commented:
Ozo,

could you please give me the exact command so that the "+" signs are not printed and that the output is printed on a file?


The conversion is much much slower than I anticipated. I would really appreciate if we would come up with a faster way?

Random Ideas:
Does the current logic skip the bits are 0's?
Can we create library's of different powers of 2 so that it does not go through the calculation for large powers, or it starts from a certain point and not from 2^1 everytime?

0
 
ozoCommented:
#!/usr/bin/perl
use Math::BigInt;
my $inputfile = "yourfile";
my $outputfile = "number.txt";
open F,"<$inputfile" or die "can't open $inputfile because $!";
binmode F;
$/=undef;
my $n=Math::BigInt->new(0);
for( unpack 'C*', <F> ){
   $n<<=8;
   $n+=$_;
}
close F;
$_ = "$n";
s/^\+//;
open O,">$outputfile" or warn "can't open $inputfile because $!";
print O;
close O;
0
 
ozoCommented:
Could you explain what you are trying to achieve by converting a large file into a single decimal number?
Perhaps we can find an easier way to accomplish the same goal?
0
 
ahoffmannCommented:
ozo,

> 3) is he right about what?
"I get the feeling that it does not use the whole file as a single number"
Means that I understand your code in that way that it reads byte by byte and append each to the previous one.
But now I see that it does not append but add. So the question is answered.

> .. converting a large file into a single decimal number?
is a huge, unique signature, somehow
0
 
SunBowCommented:
>  I am mostly interested is Win 98
- (IMO,managing large decimals frequently suffers on matters of precision)
0
 
ody13Author Commented:
Ozo,

Basically, my purpose is to be able to convert a file (in the most efficient way) to a unique number so that I can apply all sorts of equations to this unique number.

Is the latest piece of code that you wrote any faster than the previous ones?


Ahoffmann,

I am not sure I am clear on your latest comments. Do you agree with Ozo that the whole file is read?




Thanks,
Ody
0
 
ozoCommented:
convert a file (in the most efficient way) to a unique number

Then how about something like this?
#!/usr/bin/perl
my $inputfile = "yourfile";
my $outputfile = "number.txt";
open F,"<$inputfile" or die "can't open $inputfile because $!";
binmode F;
open O,">$outputfile" or warn "can't open $outfile because $!";
while( read F,$n,1 ){
    printf O "%03d",ord $n;
}
close O;
0
 
ahoffmannCommented:
> .. all sorts of equations to this unique number. ..
do you mean file1 > file2, file1 < file2, file1 == file2 ?
hmm, > and < can be done with stat(), == with a diff or cmp.
Why not simply build a hash/signature of the file?
0
 
ody13Author Commented:
Ahoffman,

I am not interested of just comparing the files (i.e. < or >). I am interested of being able to multiply, add, etc. to this unique number. Also from this unique number I would like to be able to recreate the file if needed.
0
 
ozoCommented:
while( read F,$n,1 ){  printf O "%03d",ord $n;  }
Creates a unique number which you can multiply, add etc.
You can also recreate the file with
while( read F,$n,3 ){ printf "%c", $n }
0
 
ody13Author Commented:
Ozo,

one very elementary question:

What is the command I need to use to compile a file in perl?

I 'll try your latest suggestion ASAP and let you know how it worked.

Thanks,
Odiseas
0
 
ozoCommented:
perl filename
0
 
ody13Author Commented:
Thanks Ozo,

I tried it and you are right it is very fast, much better than before!

I also tried your suggestion for recreating the original file and it worked fine with a text file, but when I tried to convert a jpg image to the unique number and then from the text file (that the unique number was printed) back to the jpg image it did not work, any clues? (the same happened with an mp3 song also)

BTW, the command that I used for that was:
while( read F,$n,3 ){ printf O "%c", $n }


Also since perl is so powerful could you please tell me how I can append to the code that you gave me some commands that would perform mathematical calculations (such as addition, multiplication, logarithms) to the output file (out1) and then output a second output file (out2) that is out1 after the calculations. I am guessing that it will be very slow performing calculations to the whole of out1 so I am open to the idea of braking it up to a number of pieces (the fewer the better)

Thanks so much!
0
 
ozoCommented:
Did you set binmode on the output file when you tried a jpeg?
#!/usr/bin/perl
my $inputfile = "number.txt";
my $outputfile = "yourfile";
open F,"<$inputfile" or die "can't open $inputfile because $!";
open O,">$outputfile" or warn "can't open $outfile because $!";
binmode O;
while( read F,$n,3 ){
     printf O "%c", $n;
}
close O;
0
 
ozoCommented:
To add two number files:
#!/usr/bin/perl
use Math::BigInt;
open F,"<out1.txt" or die $!;
$/=undef;
my $n=Math::BigInt->new(<F>);
open F,"<out2.txt" or die $!;
my $m=Math::BigInt->new(<F>);
$sum = $n+$m;
open O,">out2.txt" or warn $!;
print O $sum;
0
 
ody13Author Commented:
Setting binmode made it work.

My question was that I want to perform calculations on the number that is on out1.txt, like for example take the logarithm of that number and do other things as well and then get the output of these calculations on out2.txt.


Sorry for not being clear.
0
 
ody13Author Commented:
Hey Ozo,

you have been by far themost helpful in answering this question. As a last favor could you please take a stab on the above question?


Thanks so much,
Ody
0
 
ozoCommented:
#!/usr/bin/perl
use Math::BigInt;
open O1,"<out1.txt" or die $!;
$/=undef;
my $x=Math::BigInt->new(<O1>);
close O1;
open O2,">out2.txt" or die $!;
print O2 log(join'',map{$x->digit(-$_)}(1..16)) +($x->length()-16)*log(10);
0
 
ody13Author Commented:
Ozo,

I am getting the following error:

Can't locate object method "digit" via package "Math::BigInt" (perhaps you forgot to load "Math::BigInt"?) at cal_conversion.txt line 8

Thanks,
Ody
0
 
ody13Author Commented:
Any clues regarding the error that I am getting?
0
 
ody13Author Commented:
Any clues regarding the error that I am getting?

Thanks so much for all your help,
Ody
0
 
ozoCommented:
It sounds like you have an older version of Math::BigInt.
Try:
#!/usr/bin/perl
use Math::BigInt;
open O1,"<out1.txt" or die $!;
$/=undef;
my $x=Math::BigInt->new(<O1>);
close O1;
open O2,">out2.txt" or die $!;
($m,$e) = $" =~ /(\d{1,16})(.*)/;
print O2 (log $m)+(length $e)*log(10);
0
 
ozoCommented:
($m,$e) = $x =~ /(\d{1,16})(.*)/;
0
 
ody13Author Commented:
Ozo,

I used the above script for a file that contained the following number: "100"

Since the script takes the log of that file shouldn't I had gotten "2" in the output file?

Instead I got "4.60517018598809". Anyhow, I am mostly interested to know what do I need to change to the script so that from that number I can get the original number (i.e. "100")

Thanks
0
 
ozoCommented:
4.605170185988091 is log base e of 100
exp(4.605170185988091) will get back to 100
But this will only be approximate for larger numbers
0
 
ody13Author Commented:
Instead of taking the log is there another math calculation that you would recommend that would allow us to get back to the exact "initial" number?
0
 
ozoCommented:
There are many possible reversible transforms, depending on what you want to do.
One simple transform could be to use tr///
0
 
ody13Author Commented:
What I want to do is convert the file out1.txt to a small number that will be printed in out2.txt in the most efficient/quick way. Then I want to be able to get back to out1.txt from out2.txt again in a fast way.

Do you mind writing the code with the transform that you believe is the most efficient?

Thanks for all your help.
0
 
ozoCommented:
How small a number?  The number in out2.txt will have to be about the same size as the number in out1.txt
The most efficient/quick way is to make out1.txt and out2.txt identical.
Another simple transform might be to simply add 1 to the number in out1.txt to obtain out2.txt and subtract 1 to get back to out1.txt
I could write code to do these transforms if you wish,
but I'm not sure what goal you are trying to accomplish by doing this.
0
 
ody13Author Commented:
My goal is to have out2.txt be as small in size as possible, but also be a number that would allow me to get to precisely out1.txt

Thanks
0
 
ozoCommented:
#!/usr/bin/perl
open A,"<out1.txt" or die "can't open out1.txt $!";
$/=undef;
$_ = <A>;
tr/0-9//dc;
open B,">out2.txt" or die "can't open out2.txt $!";
$x=substr($_,0,(length)%7||7,"");
print B substr pack("N",$x),-3;
while( $x = substr($_,0,7,"") ){
    print B substr pack("N",$x),-3;
}
close B;


#!/usr/bin/perl
open B,"<out2.txt" or die "can't open out2.txt $!";
$/=\3;
open A,">out1.txt" or die "can't open out1.txt $!";
print A unpack("N","\0".<B>);
while( <B> ){
    printf A "%07d",unpack("N","\0".$_);
    print A "\n";
}
close A;
0
 
ody13Author Commented:
This is fantastic! This is exactly what I wanted.

I would love to pick your mind for different transforms that the size of out2.txt can become even smaller. Should I post that as a seperate question? Do you think you will be able to help?

Thanks again
0
 
ozoCommented:
I can get out2.txt down to 5/12 the size of out1.txt but to do much more than that would require knowing a lot more about the structure of out1.txt.  For example, are there long series of repeated zeros as in your initial example "0000000000010100"?
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

  • 23
  • 21
  • 5
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now