SQL Query comparing Float to Decimal Help

sbornstein2
sbornstein2 used Ask the Experts™
on
Hello all,

I want to make sure I am doing this the most efficient way.   I am comparing in a query a nullable float amount to a nullable decimal amount.  Right now I seem to have to do this in order for the compare to work correct.

AND LTRIM(RTRIM(ISNULL(CFS.Amount,0))) = LTRIM(RTRIM(CAST(ISNULL(CDC.Amount,0) AS decimal(18,2))))

The trimming seemed to be needed as well as latest I realized I needed an ISNULL check to 0 as well.   Is there a better way to do this?  No idea why I needed the trim but it seemed to need it or things were being missed.   I have the values coming from two different sources one is using a float and one a decimal(18,2).

Thanks all
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
View Solution Only
Senior Developer
Commented:
Why using LTRIM and RTRIM when you have FLOAT and DECIMAL/NUMERIC? These are string functions..

Then: Comparing float values for equality is, well, error prone due to the nature of floats. Under normal circumstances you need to test for ABS(float1- float2) < σ. σ (sigma) is a boundary where you consider two floats to be equal. So you need to choose it carefully.

E.g.

DECLARE @CdcAmount DECIMAL(18, 4) = 1 / 3.14159265 + 1 / 2.71828183;
DECLARE @CfsAmount FLOAT = 1 / 3.14159265 + 1 / 2.71828183;
DECLARE @SIGMA FLOAT = 0.0001;

SELECT  @CdcAmount ,
        @CfsAmount ,
        IIF(@CdcAmount = @CfsAmount, 1, 0) ,
        ABS(@CdcAmount - @CfsAmount) ,
        IIF(ABS(@CdcAmount - @CfsAmount) < @SIGMA, 1, 0);

SET @SIGMA = 0.00000001;

SELECT  @CdcAmount ,
        @CfsAmount ,
        IIF(@CdcAmount = @CfsAmount, 1, 0) ,
        ABS(@CdcAmount - @CfsAmount) ,
        IIF(ABS(@CdcAmount - @CfsAmount) < @SIGMA, 1, 0);

Open in new window

sbornstein2

Author

Commented:
Not sure why but I know without the trims for some reason comparing both in some cases was not matching.
ste5anSenior Developer

Commented:
Without actual case showing that you need that TRIMs drop them. E.g.

SELECT  LTRIM(STR(98235.12, 10, 2)) ,
        '"' + STR(98235.12, 10, 2) + '"';

Open in new window


But as you said your types are FLOAT and DECIMAL, then it does not apply.
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Find out now!
John EsraeloDatabase / SQL Developer

Commented:
compare_data_types.pngSee attached, I have made couple of comparisons.
those are nullable fields, however, did not make any of those as null to compare..
but, that's something that you can play with and see how far you can take that ..
ste5anSenior Developer

Commented:
@John, please post code, not images..
John EsraeloDatabase / SQL Developer

Commented:
I believe that typing 6 lines of code is not that cumbersome..
I get your point though.. perhaps in the near future postings.
sbornstein2

Author

Commented:
Thanks guys I am going through these this AM.
sbornstein2

Author

Commented:
Hey guys so this is what I am seeing.  Notice the .8 and .80 in the data.  Also notice the NULL issue with the AmountUSD Null compare.

Sample:

  CREATE TABLE #TempCustodianMVData
	   (
		  PortfolioID int not null,
		  MVDate datetime not null,
		  Amount float null,
		  AmountUSD float null,
		  ReconcileFlag char(1) null,
		  CurrencyID int not null
	   )

INSERT INTO #TempCustodianMVData
(
PortfolioID, 
MVDate, 
Amount,
AmountUSD, 
ReconcileFlag,
CurrencyID
)
VALUES
(
1775,
'5/31/2015',
39619139.8,
NULL,
'F',
8
)

 CREATE TABLE #TempCustodianMVSnap
	   (
		  PortfolioID int not null,
		  MVDate datetime not null,
		  Amount decimal(18,2) null,
		  AmountUSD decimal(18,2) null,
		  ReconcileFlag char(1) null,
		  CurrencyID int not null
	   )
	   
  INSERT INTO #TempCustodianMVSnap
(
PortfolioID, 
MVDate, 
Amount,
AmountUSD, 
ReconcileFlag,
CurrencyID
)
VALUES
(
1775,
'5/31/2015',
39619139.80,
NULL,
'F',
8
)

SELECT PortfolioID, 
			   MVDate, 
			   Amount,
			   AmountUSD, 
			   ReconcileFlag,
			   CurrencyID	   
		FROM #TempCustodianMVData CMV
		WHERE CMV.ReconcileFlag IN ('F','R') AND NOT EXISTS
			  (SELECT * FROM #TempCustodianMVSnap MVS 
						WHERE MVS.MVDate = CMV.MVDate
							  AND LTRIM(RTRIM(MVS.Amount)) = LTRIM(RTRIM(CAST(CMV.Amount AS decimal(18,2))))
							  AND LTRIM(RTRIM(MVS.AmountUSD)) = LTRIM(RTRIM(CAST(CMV.AmountUSD AS decimal(18,2))))
							  AND MVS.ReconcileFlag = CMV.ReconcileFlag
							  AND MVS.CurrencyID = CMV.CurrencyID)

Open in new window


This is returning a record where it should not.  If I take out the AmountUSD compare then it works where I don't return a record.   If I do this it works and does not return a record:

AND LTRIM(RTRIM(ISNULL(MVS.AmountUSD,0))) = LTRIM(RTRIM(CAST(ISNULL(CMV.AmountUSD,0) AS decimal(18,2))))

The trims also were needed for another issue I saw but can't remember where.
ste5anSenior Developer

Commented:
.8 is equal to .80. Thus it's correct.

You cannot compare NULL with the equal operator. Unknown cannot be equal to Unknown. You need the IS NULL operator.

Don't use the TRIMs when you don't know why you need it..

E.g.

SET DATEFORMAT MDY;

DECLARE @TempCustodianMVData TABLE
    (
      PortfolioID INT NOT NULL ,
      MVDate DATETIME NOT NULL ,
      Amount FLOAT NULL ,
      AmountUSD FLOAT NULL ,
      ReconcileFlag CHAR(1) NULL ,
      CurrencyID INT NOT NULL
    );

INSERT  INTO @TempCustodianMVData
VALUES  ( 1775, '5/31/2015', 39619139.8, NULL, 'F', 8 );

DECLARE @TempCustodianMVSnap TABLE
    (
      PortfolioID INT NOT NULL ,
      MVDate DATETIME NOT NULL ,
      Amount DECIMAL(18, 2) NULL ,
      AmountUSD DECIMAL(18, 2) NULL ,
      ReconcileFlag CHAR(1) NULL ,
      CurrencyID INT NOT NULL
    );
	   
INSERT  INTO @TempCustodianMVSnap
VALUES  ( 1775, '5/31/2015', 39619139.80, NULL, 'F', 8 );

SELECT  PortfolioID ,
        MVDate ,
        Amount ,
        AmountUSD ,
        ReconcileFlag ,
        CurrencyID
FROM    @TempCustodianMVData CMV
WHERE   NOT EXISTS ( SELECT *
                     FROM   @TempCustodianMVSnap MVS
                     WHERE  MVS.MVDate = CMV.MVDate
                            AND MVS.Amount = CMV.Amount
                            AND ( MVS.AmountUSD = CMV.AmountUSD
                                  OR MVS.AmountUSD IS NULL
                                  OR CMV.AmountUSD IS NULL
                                )
                            AND MVS.ReconcileFlag = CMV.ReconcileFlag
                            AND MVS.CurrencyID = CMV.CurrencyID );

Open in new window

sbornstein2

Author

Commented:
ste5an its okay to just do ISNULL(X,0) right on both sides?
ste5anSenior Developer

Commented:
It depends on whether you want

 MVS.AmountUSD = CMV.AmountUSD
 OR MVS.AmountUSD IS NULL
 OR CMV.AmountUSD IS NULL

Open in new window


or

 MVS.AmountUSD = CMV.AmountUSD
 OR (
    MVS.AmountUSD IS NULL 
    AND 
    CMV.AmountUSD IS NULL 
)

Open in new window


In the first case it's equivalent.
sbornstein2

Author

Commented:
thanks

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial