Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

query regarding Dataset storage?

Posted on 2004-10-26
14
Medium Priority
?
398 Views
Last Modified: 2013-11-15
If we create a Dataset for each user, where will be these datasets stored.And for instance if there are 1 million users active then each user will have a separate dataset.Will this affect the performance or how will it be handled by framework?
0
Comment
Question by:pml_siva
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 3
  • +2
14 Comments
 
LVL 10

Accepted Solution

by:
jnhorst earned 500 total points
ID: 12418633
Wow, 1 million users and a dataset for each!

But seriously, I would think the only way to deal with that would be a database back end that would keep each user's data.  The other option would be saving the dataset as an XML file (the dataset class has WriteXml() and ReadXml() methods for this), naming the file according to the user's id.  If the dataset is large (lots of data), though, the disk IO cost might hit performance in an unacceptable way.

Datasets are xml abstractions of the database objects like tables and relationships, so if the technology at hand includes or can include a database like SQL Server or Oracle, it would be best to store the data in a database, and then load up/update the dataset from that data.

John
0
 
LVL 28

Expert Comment

by:mmarinov
ID: 12418681
Hi pml_siva,

I'm totaly agree with jnhorst. Also why do you have to hold all users' data in dataset in the same time ? to answer you first question - DataSet is stored in the memory - it gets all the data and stays there. For example and see the difference - the datareader object read one recod at a time and does not cosume a lot of memory resources, but it keeps the connection to the database open

Regards!
B..M
0
 
LVL 26

Expert Comment

by:Rejojohny
ID: 12418760

>>If we create a Dataset for each user
Is it just create .. i mean created in a page? do u plan to store it too? If s, how? is the data common between the users? what r u planning to use to store? cache?

>>Will this affect the performance
it will surely affect performance if u have 1 million dataset in any storage like cache, session, application etc ..what u could think of is having a common storage for all users and then querrying from it whenever required .. so this will be like a temporary storage on ur web server (memory) ...
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 4

Author Comment

by:pml_siva
ID: 12419013
How to design for this type of requirement?
For ex I'm loading Dataset in pageload of my application when the user logs in.
If 1 million user logs in at the same time,what will happen?Whether 1 million datasets will be created or is there any?If 1 million datasets created then for ex if each datasets holds a size of 1Kb then for 1 million it would occupy 1 million Kb size in the web server..Definetly this will affect the performance of the web server.So how to implement this.......
0
 
LVL 28

Assisted Solution

by:mmarinov
mmarinov earned 500 total points
ID: 12419055
pml_siva,

if you have 1 million user of your web site you must have a really good server :)
so yes, because you create dataset in the page_load - there will be 1 million datasets in the server's memory
so if you have a good configuration of your server ( with muc memory - may 2 GB ) it won't be a problem
also for operationg with the request of 1 million users you must have at least dual-processor machine

B..M
0
 
LVL 4

Author Comment

by:pml_siva
ID: 12419067
Ok.How the datasets will be stored internally in the Cache ie.In which format?
0
 
LVL 4

Author Comment

by:pml_siva
ID: 12419091
Ok For this case how to design the Implementations without affecting the performance..
When the User Logs in I need to load the Datas in the DB into the DataGrid and i need  to perform actions such as sorting etc...So if 1 million users logs in at the same time.Then it would be difficult ...and it creates 1 million datasets in the server.
So which is the right way to implement this?
Can u help me regarding this....

Thanks in Advance.
Pml_siva
0
 
LVL 28

Expert Comment

by:mmarinov
ID: 12419107
pml_siva,

because of this you can use reader object - it reads a record at a time and does not keep the records in the memory
the problem is with reader object that keeps the connection open
so you have to choose which scenario to use
for cashing user's dataset you can see this article :http://weblogs.asp.net/tatochip/archive/2004/02/09/70105.aspx

B..M
0
 
LVL 26

Assisted Solution

by:Rejojohny
Rejojohny earned 500 total points
ID: 12419136
r u sure u want to look at this scenario ..  i mean as "mmarinov" menioned abvove u really need a very good server for implementing such an application .. normally for a page to excute, the time taken is less than a second .. so u r basically taking about about 1 million connection for every 1 second !!! higly impossoble ... what kind of a application r u taking about .. now the question arises what kind of data is to be shown? is there anything common between the data shown to the user .. if s, then u could store the dataset (which will be a select * from the table) in ur application cache and then filter from here based on requirement . so basically u have just one dataset in the memory ...
0
 
LVL 17

Assisted Solution

by:AerosSaga
AerosSaga earned 500 total points
ID: 12420832
only query the records you need, use the top function heavily.  Storing all the datasets in memory pretty much kills any advantage of having one.  I agree with all the other experts this is not a good idea.

Aeros
0
 
LVL 26

Expert Comment

by:Rejojohny
ID: 12703071
I would recommend a split of points ... we have all tried to explain the cons of such an approach and also suggested alternatives ..
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Microservice architecture adoption brings many advantages, but can add intricacy. Selecting the right orchestration tool is most important for business specific needs.
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will walk an individual through the process of configuring basic necessities in order to use the 2010 version of Data Protection Manager. These include storage, agents, and protection jobs. Launch Data Protection Manager from the deskt…

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question