Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

optimize select distinct

Posted on 2010-09-23
7
Medium Priority
?
530 Views
Last Modified: 2012-05-10
Hello,

How can I optimize this select distinct :
Select      Distinct      
      TEST.SOC_CODE CODE_SOCIETE,
      TEST.CLI_CODE CODE_CLIENT,
      TEST.TEST_ORIGINE_CLIENT CODE_ORIGINE_CLIENT,
      TEST.AGE_CODE CODE_AGENCE,
      TEST.TEST_CODE NUMERO_TESTIS,
      TEST.TEST_VERSION CODE_VERSION_TESTIS,
      case when TEST.TEST_NATURE_LIVRAISON = 'L' then 'LIVRE' else case when TEST.TEST_NATURE_LIVRAISON = 'E' then 'A ENLEVER' else null end end NATURE_LIVRAISON,
      case when TEST.TEST_MODE_VENTE = 'M' then 'MAGASIN' else case when TEST.TEST_MODE_VENTE = 'D' then 'DIRECT' else null end end MODE_VENTE,
      case when TEST.TEST_ETAT = 'I' then 'A RELANCER' else case when TEST.TEST_ETAT = 'R' then 'ABOUTI PARTIEL' else null end end ETAT_TESTIS,
      cast(datepart("year", TEST.TEST_DATE) * 10000 + datepart("month", TEST.TEST_DATE) * 100  + datepart("day", TEST.TEST_DATE) as varchar) DATE_TESTIS,
      cast(datepart("year", TEST.TEST_DATE_PREVUE_LIVR) * 10000 + datepart("month", TEST.TEST_DATE_PREVUE_LIVR) * 100  + datepart("day", TEST.TEST_DATE_PREVUE_LIVR) as varchar) DATE_VALIDITE,
      cast(datepart("year", TEST.TEST_DATE_SOUHAITEE) * 10000 + datepart("month", TEST.TEST_DATE_SOUHAITEE) * 100  + datepart("day", TEST.TEST_DATE_SOUHAITEE) as varchar) DATE_TRAVAUX,
      convert(varchar, TEST.TEST_MNT_TOT_HT_BON_PX_DEP) MONTANT_TOTAL_HT_PRIX_DEPART,
      convert(varchar, TEST.TEST_TOT_HT_FRAIS_TRANSP) TOTAL_HT_FRAIS_TRANSPORTS_PRODUITS
From      ENT_TESTIS as TEST, TECH_REGION as REGION, TECH_EXTRACT_CLOE as EXTRACT_CLOE, ENT_LIEN_ATC_CLI_ENCOURS as ACE
Where       (1=1)
And (REGION.TRE_ACTIF='O')
 And (TEST.TEST_TYPE IS NOT NULL AND TEST.TEST_TYPE = 'D' AND
TEST.TEST_ETAT in ('I','R') AND
(TEST.TEST_ECARTE_RELANCE IS NULL OR (TEST.TEST_ECARTE_RELANCE IS NOT NULL AND TEST.TEST_ECARTE_RELANCE != 'O')) AND
TEST.TEST_DATE_PREVUE_LIVR IS NOT NULL AND TEST.TEST_DATE_PREVUE_LIVR >= dateadd("month", -3, CURRENT_TIMESTAMP) AND
TEST.TEST_TECH_DATE_SUPP IS NULL AND TEST.TEST_TECH_DATE_SUPP_CLOE IS NULL)
 And (Coalesce(TEST.TEST_TECH_DATE_MAJ,TEST.TEST_TECH_DATE_CREA) > EXTRACT_CLOE.TEC_DATE_EXTRACT
OR
Coalesce(ACE.ACE_TECH_DATE_MAJ,ACE.ACE_TECH_DATE_CREA) > EXTRACT_CLOE.TEC_DATE_EXTRACT)
 And (EXTRACT_CLOE.TEC_GROUPE='TEST')
 And (ACE.ACE_ORIGINE_CLIENT = 'S')
 And (TEST.REG_CODE=REGION.TRE_CODE)
AND (TEST.REG_CODE=EXTRACT_CLOE.TEC_REG_CODE)
AND (TEST.SOC_CODE=ACE.SOC_CODE AND TEST.CLI_CODE=ACE.CLI_CODE)


Thanks
bibi
0
Comment
Question by:bibi92
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
7 Comments
 
LVL 29

Expert Comment

by:QPR
ID: 33742639
have you considered the sql 2008 query optimizer
0
 
LVL 29

Expert Comment

by:QPR
ID: 33742654
0
 
LVL 50

Expert Comment

by:Lowfatspread
ID: 33743502
try this

what is the main current problem?

what indexes are available / being used?

how many rows are returned?
how many rows on each table?

are you certain you need the distinct?
 what order do you want the results returned in ?

 NB you do not have to specify
        x is not null and x <> 'B'
      since x <> 'B'   will ignore null values anyway




 
 
Select      Distinct      
      TEST.SOC_CODE CODE_SOCIETE,
      TEST.CLI_CODE CODE_CLIENT,
      TEST.TEST_ORIGINE_CLIENT CODE_ORIGINE_CLIENT,
      TEST.AGE_CODE CODE_AGENCE,
      TEST.TEST_CODE NUMERO_TESTIS,
      TEST.TEST_VERSION CODE_VERSION_TESTIS,
      case TEST.TEST_NATURE_LIVRAISON when 'L' then 'LIVRE' when 'E' then 'A ENLEVER' else null end NATURE_LIVRAISON,
      case TEST.TEST_MODE_VENTE when 'M' then 'MAGASIN' when 'D' then 'DIRECT' else null end MODE_VENTE,
      case TEST.TEST_ETAT when 'I' then 'A RELANCER' else when 'R' then 'ABOUTI PARTIEL' else null end ETAT_TESTIS,
      convert(char(8),TEST.TEST_DATE ,112) DATE_TESTIS,
      convert(char(8),TEST.TEST_DATE_PREVUE_LIVR,112) DATE_VALIDITE,
      convert(char(8),TEST.TEST_DATE_SOUHAITEE,112) DATE_TRAVAUX,
      convert(varchar, TEST.TEST_MNT_TOT_HT_BON_PX_DEP) MONTANT_TOTAL_HT_PRIX_DEPART,
      convert(varchar, TEST.TEST_TOT_HT_FRAIS_TRANSP) TOTAL_HT_FRAIS_TRANSPORTS_PRODUITS
From      ENT_TESTIS as TEST
inner join (select tre_code 
              from TECH_REGION 
             Where TRE_ACTIF='O'             
            )    as REGION
   ON TEST.REG_CODE=REGION.TRE_CODE
Inner Join (select tec_reg_code,tec_date_extract 
     from TECH_EXTRACT_CLOE
    Where TEC_GROUPE='TEST'
    Group by tec_reg_code,tec_date_extract    
   )    as EXTRACT_CLOE
 On TEST.REG_CODE=EXTRACT_CLOE.TEC_REG_CODE  
Inner Join (select Distinct soc_code,cli_code
                  ,coalesce(ace_tec_date_maj,ace_tech_date,crea) as Acedate 
     from ENT_LIEN_ATC_CLI_ENCOURS
    Where ace_origine_client='S' 
   )  as ACE
  On TEST.SOC_CODE=ACE.SOC_CODE
 AND TEST.CLI_CODE=ACE.CLI_CODE
   
Where 
  TEST.TEST_TYPE = 'D' 
       AND TEST.TEST_ETAT in ('I','R') 
       AND(TEST.TEST_ECARTE_RELANCE IS NULL               
             or TEST.TEST_ECARTE_RELANCE != 'O'
           )
       AND TEST.TEST_DATE_PREVUE_LIVR >= dateadd("month", -3, CURRENT_TIMESTAMP) 
       AND TEST.TEST_TECH_DATE_SUPP IS NULL 
       AND TEST.TEST_TECH_DATE_SUPP_CLOE IS NULL
     
 And (Coalesce(TEST.TEST_TECH_DATE_MAJ,TEST.TEST_TECH_DATE_CREA) > EXTRACT_CLOE.TEC_DATE_EXTRACT
       OR Ace.Acedate > EXTRACT_CLOE.TEC_DATE_EXTRACT
     )
 

Open in new window

0
Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

 

Author Comment

by:bibi92
ID: 33744794
Hi,
200000 ROWS are returned
Indexes are created on PK.
Tables are nearly 1000000rows.
The use of distinct is necessary for the customer.
The order is not indispensable.

Thanks
bibi
0
 
LVL 50

Accepted Solution

by:
Lowfatspread earned 2000 total points
ID: 33751789
1) are all 200K rows required?
2) what is actually making the rows not distinct?
          1:m relationships... , removal of the time component of the dates, ....?

3) Do all tables contain 1 million rows?
      if not what are the actual table sizes

4) what are the tables primary keys?
    are the PK indexes clustered?
    any reason these are the only indexes?
    how are the tables used... (what sort of system is this .. OLTP,MIS,OLAP, mixed, ...)?

 
 
Select      Distinct      
      TEST.SOC_CODE CODE_SOCIETE,
      TEST.CLI_CODE CODE_CLIENT,
      TEST.TEST_ORIGINE_CLIENT CODE_ORIGINE_CLIENT,
      TEST.AGE_CODE CODE_AGENCE,
      TEST.TEST_CODE NUMERO_TESTIS,
      TEST.TEST_VERSION CODE_VERSION_TESTIS,
      case TEST.TEST_NATURE_LIVRAISON when 'L' then 'LIVRE' when 'E' then 'A ENLEVER' else null end NATURE_LIVRAISON,
      case TEST.TEST_MODE_VENTE when 'M' then 'MAGASIN' when 'D' then 'DIRECT' else null end MODE_VENTE,
      case TEST.TEST_ETAT when 'I' then 'A RELANCER' when 'R' then 'ABOUTI PARTIEL' else null end ETAT_TESTIS,
      convert(char(8),TEST.TEST_DATE ,112) DATE_TESTIS,
      convert(char(8),TEST.TEST_DATE_PREVUE_LIVR,112) DATE_VALIDITE,
      convert(char(8),TEST.TEST_DATE_SOUHAITEE,112) DATE_TRAVAUX,
      convert(varchar, TEST.TEST_MNT_TOT_HT_BON_PX_DEP) MONTANT_TOTAL_HT_PRIX_DEPART,
      convert(varchar, TEST.TEST_TOT_HT_FRAIS_TRANSP) TOTAL_HT_FRAIS_TRANSPORTS_PRODUITS
From      ENT_TESTIS as TEST
Inner Join (select a.tec_reg_code,a.tec_date_extract 
              from TECH_EXTRACT_CLOE as a
             Inner Join TECH_REGION as b  
                on a.tec_reg_code=b.tre_code
             Where a.TEC_GROUPE='TEST'
               and b.TRE_ACTIF='O'
             Group by a.tec_reg_code,a.tec_date_extract    
           )    as EXTRACT_CLOE
 On TEST.REG_CODE=EXTRACT_CLOE.TEC_REG_CODE  
Inner Join (select Distinct soc_code,cli_code
                  ,coalesce(ace_tec_date_maj,ace_tech_date,crea) as Acedate 
     from ENT_LIEN_ATC_CLI_ENCOURS
    Where ace_origine_client='S' 
   )  as ACE
  On TEST.SOC_CODE=ACE.SOC_CODE
 AND TEST.CLI_CODE=ACE.CLI_CODE
   
Where 
  TEST.TEST_TYPE = 'D' 
       AND TEST.TEST_ETAT in ('I','R') 
       AND(TEST.TEST_ECARTE_RELANCE IS NULL               
             or TEST.TEST_ECARTE_RELANCE != 'O'
           )
       AND TEST.TEST_DATE_PREVUE_LIVR >= dateadd("month", -3, CURRENT_TIMESTAMP) 
       AND TEST.TEST_TECH_DATE_SUPP IS NULL 
       AND TEST.TEST_TECH_DATE_SUPP_CLOE IS NULL
     
 And (Coalesce(TEST.TEST_TECH_DATE_MAJ,TEST.TEST_TECH_DATE_CREA) > EXTRACT_CLOE.TEC_DATE_EXTRACT
       OR Ace.Acedate > EXTRACT_CLOE.TEC_DATE_EXTRACT
     )
 

Open in new window

0
 

Author Comment

by:bibi92
ID: 33765886
Hello,

Thanks

1) are all 200K rows required? ---> Yes
2) what is actually making the rows not distinct? ---> dates
          1:m relationships... , removal of the time component of the dates, ....?

3) Do all tables contain 1 million rows? ---> Yes
      if not what are the actual table sizes

4) what are the tables primary keys?
    are the PK indexes clustered? ---> Yes
    any reason these are the only indexes? I don't know. The customer can't answer to this question
    how are the tables used... ---> OLTP

Regards
bibi
0
 

Author Closing Comment

by:bibi92
ID: 33952962
Thanks bibi
0

Featured Post

How Blockchain Is Impacting Every Industry

Blockchain expert Alex Tapscott talks to Acronis VP Frank Jablonski about this revolutionary technology and how it's making inroads into other industries and facets of everyday life.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This is basically a blog post I wrote recently. I've found that SARGability is poorly understood, and since many people don't read blogs, I figured I'd post it here as an article. SARGable is an adjective in SQL that means that an item can be fou…
In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…
Have you created a query with information for a calendar? ... and then, abra-cadabra, the calendar is done?! I am going to show you how to make that happen. Visualize your data!  ... really see it To use the code to create a calendar from a q…

661 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question