Help with select query speed

I have a process that helps me organize data into groups.  Below is a select that helps pull out certain characteristics of the product listings.  I am averaginges 5-8 seconds per query, the problem is, is that there are about 10,000 products that it needs to search through another table of listings that has about 1.6 million records. So this process can take 10+ hours to complete.  All the columns here are indexed.  Any suggestions?

select item_id, item_title,end_date
      from productlistings
      where end_date > '2012-03-01 00:00:00'
    and end_date < '2012-03-31 23:59:59'
    and (item_title like '% 10114 %')
      and (item_title not like '% only %')
      and (item_title not like '% only')
      and (item_title not like '% lot %')
      and (item_title not like 'lot %')
      and (item_title not like '%lot')
      and (item_title not like 'loose %')
      and (item_title not like '% loose')
      and (item_title not like '% sealed box %')
      and (item_title not like '% packs %')
      and (item_title not like '% boxcase %')
      and (item_title not like '% 60 %')
      and (item_title not like '% complete set %')
      and (item_title not like '% all 16 packs')
      and (item_title not like '% case')
      and (item_title not like '% set of 16 %')
      and (item_title not like '% set 16 %')
      and (item_title not like '% all 16 %')
      and (item_title not like '% set of 16')
      and (item_title not like '% loose %')
      and (item_title not like 'set of 16 %')
      and (item_title not like '% set of %')
      and (item_title not like '% custom %')
        and set_id2 = 0
      and cleared_ind = 'x'
Who is Participating?
Aaron TomoskyConnect With a Mentor Technology ConsultantCommented:
If the date filter significantly reduces the recordset size, you can try a subquery with that as the only filter then apply the rest.

select item_id, item_title,end_date
      from (select * from productlistings
      where end_date between '2012-03-01' and  '2012-04-01') as Pl
    Where (item_title like '% 10114 %')
      and (item_title not like '% only %')

If any of your where filters take out a big chunk of stuff, a subquery can significantly speed things up. You can also use cte , I just find that harder to read.

Also you have duplication going on:
      and (item_title not like '% lot %')
      and (item_title not like 'lot %')
      and (item_title not like '%lot')

% will match any OR no characters so the second and third statements above are pointless. Just removing all those may help.
Darius GhassemCommented:
Do the product listing have a product_listing_id for example. Can you filter a different way?
theideabulbAuthor Commented:
no not really, think of it like an ebay feed or craiglists feed. It is going through the titles and looking for specific keyword, and filtering out those words.  We get an update every month that adds an extra 70-100k in listings to filter.   I am trying to grab the most recent date, but I still need to leave all the old listings in for individual searches.
Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

LIKE or NOT LIKE '% keyword %' does not make use of indexes.  Consider using boolean full-text search.  See

Note that this is only available for myisam tables, and from what I read it doesn't really scale very well for very large data sets.  If you hit a performance issue with it a good alternative is a 3rd-party search engine like sphinx ( or maybe you can just integrate google search to your app.
Tomas Helgi JohannssonConnect With a Mentor Commented:

Put this index on your table and see if that doesn't help you
decreasing the query time

create index prodidcleard_ix
using btree
on productlistings (set_id2, cleared_ind, end_date,item_title);

      Tomas Helgi
theideabulbAuthor Commented:
Combined, both of these comments shaved off 2-3 seconds, which adds up over the 10K plus times this will run.  Thank you for your knowledge.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.