Subtracting 2 Numbers Problem

Posted on 2008-11-14
Last Modified: 2013-11-13
I am trying to subtract 2 numbers; the numbers come from different sources (one from the Internet, the other from a table in a database where its type is decimal.

If they both have the same value, when I try to subtract them, I get a result like 7.105427357601e-015. I tried casting to int (.to_i) and it works fine but I lose the decimal part.  If I cast them to float I get a similar result to that given above.  I cannoy think of any other cast to make.

Another thing if the two numbers are 84.21 and 80.21, the difference is 3.9999999999999

Subtracting 84.21 from 80.18 produces the correct value.

Sometimes I despair.

Can anyone help?

Question by:witgrefe
    LVL 60

    Expert Comment

    by:Geert Bormans
    7.105427357601e-015 is a very, very, very small number,
    so this is a precision issue

    I assume that you don't want to loose the decimal part,
    so you must have some precision in mind.

    What you could do is multiply by 100, then round and the divide by 100 again.
    Then you will have a precison up to 0.01

    3.9999999999999 will become 4.00
    and 7.105427357601e-015  will become 0

    If you need more precision, multiply by thousand, round and divide again
    LVL 60

    Expert Comment

    by:Geert Bormans
    Try this

    result = 3.99999999999
    puts (result * 100).round.div(100).to_s


    result = 7.105427357601e-015
    puts (result * 100).round.div(100).to_s
    LVL 60

    Expert Comment

    by:Geert Bormans
    I ran into two traps here
    div gives an integer result... that is not what you want
    / would return a truncated integer since the first argument is an integer
    so, I need explicit casting to float first

    puts ((result * 100).round.to_f / 100).to_s

    Author Comment

    But I need 2 decimal places, what you have suggested gives me no decimal places.  It is fine if for the 2 examples I gave, but if I subtract 80.55 from 85.7, I get  5 without any decimal place
    LVL 60

    Accepted Solution

    Yep, I spotted the error
    please look at my last comment

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    Join & Write a Comment

    I recently rediscovered rails when I needed a holiday project and decided to build a management dashboard for the company where I work.  With it being a project done in my free time, I could focus my time on learning the basics rather than trying to…
    In Ruby, Call or invoke a API DLL library is easily via Win32API class, win32-api gem or other gems. For general DLL API call, there are quite a few references, some good tips list below:…
    To add imagery to an HTML email signature, you have two options available to you. You can either add a logo/image by embedding it directly into the signature or hosting it externally and linking to it. The vast majority of email clients display l…
    This video discusses moving either the default database or any database to a new volume.

    728 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now