troubleshooting Question

SQLite max query

Avatar of Sam K
Sam KFlag for Australia asked on
Databases.NET ProgrammingC#SQL
6 Comments1 Solution90 ViewsLast Modified:
Hello experts,
I was wondering if there was a better method of creating a SQL search parameters than what I have used.

In my application written in C# connected to an SQLite database, I have provided a single column datagrid for users to enter values, my original intention was for small to medium scale queries for quick results. As I am sure everyone has found out that a developers expectations and what a user does is often very different.
The program currently opens a connection to the DB and loops through the values checking for a scalar value, if no scalar, it reports the selected value doesn't exist and continues to the end of the loop and closes the connection, alerting the user to the erroneous value/s.
It then opens a new connection to the DB where it creates an SQL query string in a list, with a new item added for each non-blank entry in the datagrid , because the values exist in two separate tables (for unfortunate legacy reasons) Views are created and destroyed in the statement.

A sample query for 8 items would look like the following:

CREATE VIEW view1 AS SELECT * FROM table1, table2;
CREATE VIEW view2 AS SELECT * FROM table3, table4;
CREATE VIEW collection AS SELECT * FROM view2
UNION ALL
SELECT * FROM view1;
SELECT * FROM collection WHERE collection.Column1 LIKE "%value1%"
or collection.Column1 like "%value2%"
or collection.Column1 like "%value3%"
or collection.Column1 like "%value4%"
or collection.Column1 like "%value5%"
or collection.Column1 like "%value6%"
or collection.Column1 like "%value7%"
or collection.Column1 like "%value8%"
DROP VIEW view1;
DROP VIEW view2;
DROP VIEW collection;

Whilst the process time for this example in Navicat is 0.304 seconds, when users enter hundreds or even thousands of items, the program becomes slow, unresponsive and it turns out that the SQLite engine can only handle a max of 1000 requests per query.
For each value in the datagrid a "or collection like value.." is created in a list which is joined together using using a list.toarray ie: string s1 = string.Join(string.empty, list.ToArray()).

Is there a way to make this part of the program more robust and efficient?
Thank you
ASKER CERTIFIED SOLUTION
Join our community to see this answer!
Unlock 1 Answer and 6 Comments.
Start Free Trial
Learn from the best

Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.

Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 6 Comments.
Try for 7 days

”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.

-Mike Kapnisakis, Warner Bros