asked on
Hello experts,
I was wondering if there was a better method of creating a SQL search parameters than what I have used.
In my application written in C# connected to an SQLite database, I have provided a single column datagrid for users to enter values, my original intention was for small to medium scale queries for quick results. As I am sure everyone has found out that a developers expectations and what a user does is often very different.
The program currently opens a connection to the DB and loops through the values checking for a scalar value, if no scalar, it reports the selected value doesn't exist and continues to the end of the loop and closes the connection, alerting the user to the erroneous value/s.
It then opens a new connection to the DB where it creates an SQL query string in a list, with a new item added for each non-blank entry in the datagrid , because the values exist in two separate tables (for unfortunate legacy reasons) Views are created and destroyed in the statement.
A sample query for 8 items would look like the following:
CREATE VIEW view1 AS SELECT * FROM table1, table2;
CREATE VIEW view2 AS SELECT * FROM table3, table4;
CREATE VIEW collection AS SELECT * FROM view2
UNION ALL
SELECT * FROM view1;
SELECT * FROM collection WHERE collection.Column1 LIKE "%value1%"
or collection.Column1 like "%value2%"
or collection.Column1 like "%value3%"
or collection.Column1 like "%value4%"
or collection.Column1 like "%value5%"
or collection.Column1 like "%value6%"
or collection.Column1 like "%value7%"
or collection.Column1 like "%value8%"
DROP VIEW view1;
DROP VIEW view2;
DROP VIEW collection;
Whilst the process time for this example in Navicat is 0.304 seconds, when users enter hundreds or even thousands of items, the program becomes slow, unresponsive and it turns out that the SQLite engine can only handle a max of 1000 requests per query.
For each value in the datagrid a "or collection like value.." is created in a list which is joined together using using a list.toarray ie: string s1 = string.Join(string.empty, list.ToArray()).
Is there a way to make this part of the program more robust and efficient?
Thank you
I was wondering if there was a better method of creating a SQL search parameters than what I have used.
In my application written in C# connected to an SQLite database, I have provided a single column datagrid for users to enter values, my original intention was for small to medium scale queries for quick results. As I am sure everyone has found out that a developers expectations and what a user does is often very different.
The program currently opens a connection to the DB and loops through the values checking for a scalar value, if no scalar, it reports the selected value doesn't exist and continues to the end of the loop and closes the connection, alerting the user to the erroneous value/s.
It then opens a new connection to the DB where it creates an SQL query string in a list, with a new item added for each non-blank entry in the datagrid , because the values exist in two separate tables (for unfortunate legacy reasons) Views are created and destroyed in the statement.
A sample query for 8 items would look like the following:
CREATE VIEW view1 AS SELECT * FROM table1, table2;
CREATE VIEW view2 AS SELECT * FROM table3, table4;
CREATE VIEW collection AS SELECT * FROM view2
UNION ALL
SELECT * FROM view1;
SELECT * FROM collection WHERE collection.Column1 LIKE "%value1%"
or collection.Column1 like "%value2%"
or collection.Column1 like "%value3%"
or collection.Column1 like "%value4%"
or collection.Column1 like "%value5%"
or collection.Column1 like "%value6%"
or collection.Column1 like "%value7%"
or collection.Column1 like "%value8%"
DROP VIEW view1;
DROP VIEW view2;
DROP VIEW collection;
Whilst the process time for this example in Navicat is 0.304 seconds, when users enter hundreds or even thousands of items, the program becomes slow, unresponsive and it turns out that the SQLite engine can only handle a max of 1000 requests per query.
For each value in the datagrid a "or collection like value.." is created in a list which is joined together using using a list.toarray ie: string s1 = string.Join(string.empty, list.ToArray()).
Is there a way to make this part of the program more robust and efficient?
Thank you
Learn from the best
Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.
Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 6 Comments.
Try for 7 days”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.
Our community of experts have been thoroughly vetted for their expertise and industry experience.
The Most Valuable Expert award recognizes technology experts who passionately share their knowledge with the community, demonstrate the core values of this platform, and go the extra mile in all aspects of their contributions. This award is based off of nominations by EE users and experts. Multiple MVEs may be awarded each year.
This award recognizes a member of Experts Exchange who has made outstanding contributions to the community within their first year as an expert. The Rookie of the Year is awarded to a new expert who has the highest number of quality contributions.