Solved

What is best way to store high frequency changing and high frequency query data for web user?

Posted on 2014-02-21
4
458 Views
Last Modified: 2014-03-21
I have stock price data that
- many users will queries quite often
- about 2000-3000 rows at times per symbol
- yet data appending pretty fast.

what would be the best way to tackle this problem.
- sql is currently has high cpu because users keep repeating requests
- sometime data change fast sometime not - but users have habit for repeat queries
- I am thinking about making data available on memory entirely but
-- Can we make sql on memory entirely and will this help my situation
-- If not using sql but create cache in memory - what would be the best way so web user can see the price rows and most up to date price rows without much cpu per calling

Many thanks,
0
Comment
Question by:JSW21
  • 2
4 Comments
 
LVL 65

Assisted Solution

by:Jim Horn
Jim Horn earned 250 total points
Comment Utility
How often is this data updated?  

As long as we're not talking realtime, one solution would be to create a couple of materialized views, indexed on the columns that users regularly search.  

Then in the front end, have it interpret which term(s) the users entered as search criteria, and call the appropriate view.

Then after the nightly backup/restore create a job that drops and re-executes these views.
0
 
LVL 69

Accepted Solution

by:
ScottPletcher earned 250 total points
Comment Utility
The single best way to address is to create the best clustered index possible on the table, one that resolves the majority of common lookups.

Unfortunately, I don't know enough detail about your table to assist with that at this point.

But you need to look at the JOIN and WHERE conditions in the most common queries.  Then determine if those columns are appropriate for the clustered index, and in what order the key column(s) should be.


If you need historical stock info, an indexed view might indeed help.  SQL can, and often will, use that view even if you don't explicitly reference it in the query.
0
 

Author Comment

by:JSW21
Comment Utility
You refer to View as feature in the sql right?
So by creating view and indexed it properly will increase performance for repeated queries.

"Then after the nightly backup/restore create a job that drops and re-executes these views."

If I create a view why do I have to drop and recreate it again nightly? Would the View update automatically when data changes?
0
 
LVL 65

Expert Comment

by:Jim Horn
Comment Utility
Thanks for the split.  Good luck with your project.  -Jim
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
Via a live example, show how to extract insert data into a SQL Server database table using the Import/Export option and Bulk Insert.
Use Wufoo, an online form creation tool, to make powerful forms. Learn how to choose which pages of your form are visible to your users based on their inputs. The page rules feature provides you with an opportunity to create if:then statements for y…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now