Solved

query regarding Dataset storage?

Posted on 2004-10-26
392 Views
Last Modified: 2013-11-15
If we create a Dataset for each user, where will be these datasets stored.And for instance if there are 1 million users active then each user will have a separate dataset.Will this affect the performance or how will it be handled by framework?
0
Question by:pml_siva
    11 Comments
     
    LVL 10

    Accepted Solution

    by:
    Wow, 1 million users and a dataset for each!

    But seriously, I would think the only way to deal with that would be a database back end that would keep each user's data.  The other option would be saving the dataset as an XML file (the dataset class has WriteXml() and ReadXml() methods for this), naming the file according to the user's id.  If the dataset is large (lots of data), though, the disk IO cost might hit performance in an unacceptable way.

    Datasets are xml abstractions of the database objects like tables and relationships, so if the technology at hand includes or can include a database like SQL Server or Oracle, it would be best to store the data in a database, and then load up/update the dataset from that data.

    John
    0
     
    LVL 28

    Expert Comment

    by:mmarinov
    Hi pml_siva,

    I'm totaly agree with jnhorst. Also why do you have to hold all users' data in dataset in the same time ? to answer you first question - DataSet is stored in the memory - it gets all the data and stays there. For example and see the difference - the datareader object read one recod at a time and does not cosume a lot of memory resources, but it keeps the connection to the database open

    Regards!
    B..M
    0
     
    LVL 26

    Expert Comment

    by:Rejojohny

    >>If we create a Dataset for each user
    Is it just create .. i mean created in a page? do u plan to store it too? If s, how? is the data common between the users? what r u planning to use to store? cache?

    >>Will this affect the performance
    it will surely affect performance if u have 1 million dataset in any storage like cache, session, application etc ..what u could think of is having a common storage for all users and then querrying from it whenever required .. so this will be like a temporary storage on ur web server (memory) ...
    0
     
    LVL 4

    Author Comment

    by:pml_siva
    How to design for this type of requirement?
    For ex I'm loading Dataset in pageload of my application when the user logs in.
    If 1 million user logs in at the same time,what will happen?Whether 1 million datasets will be created or is there any?If 1 million datasets created then for ex if each datasets holds a size of 1Kb then for 1 million it would occupy 1 million Kb size in the web server..Definetly this will affect the performance of the web server.So how to implement this.......
    0
     
    LVL 28

    Assisted Solution

    by:mmarinov
    pml_siva,

    if you have 1 million user of your web site you must have a really good server :)
    so yes, because you create dataset in the page_load - there will be 1 million datasets in the server's memory
    so if you have a good configuration of your server ( with muc memory - may 2 GB ) it won't be a problem
    also for operationg with the request of 1 million users you must have at least dual-processor machine

    B..M
    0
     
    LVL 4

    Author Comment

    by:pml_siva
    Ok.How the datasets will be stored internally in the Cache ie.In which format?
    0
     
    LVL 4

    Author Comment

    by:pml_siva
    Ok For this case how to design the Implementations without affecting the performance..
    When the User Logs in I need to load the Datas in the DB into the DataGrid and i need  to perform actions such as sorting etc...So if 1 million users logs in at the same time.Then it would be difficult ...and it creates 1 million datasets in the server.
    So which is the right way to implement this?
    Can u help me regarding this....

    Thanks in Advance.
    Pml_siva
    0
     
    LVL 28

    Expert Comment

    by:mmarinov
    pml_siva,

    because of this you can use reader object - it reads a record at a time and does not keep the records in the memory
    the problem is with reader object that keeps the connection open
    so you have to choose which scenario to use
    for cashing user's dataset you can see this article :http://weblogs.asp.net/tatochip/archive/2004/02/09/70105.aspx

    B..M
    0
     
    LVL 26

    Assisted Solution

    by:Rejojohny
    r u sure u want to look at this scenario ..  i mean as "mmarinov" menioned abvove u really need a very good server for implementing such an application .. normally for a page to excute, the time taken is less than a second .. so u r basically taking about about 1 million connection for every 1 second !!! higly impossoble ... what kind of a application r u taking about .. now the question arises what kind of data is to be shown? is there anything common between the data shown to the user .. if s, then u could store the dataset (which will be a select * from the table) in ur application cache and then filter from here based on requirement . so basically u have just one dataset in the memory ...
    0
     
    LVL 17

    Assisted Solution

    by:AerosSaga
    only query the records you need, use the top function heavily.  Storing all the datasets in memory pretty much kills any advantage of having one.  I agree with all the other experts this is not a good idea.

    Aeros
    0
     
    LVL 26

    Expert Comment

    by:Rejojohny
    I would recommend a split of points ... we have all tried to explain the cons of such an approach and also suggested alternatives ..
    0

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Highfive + Dolby Voice = No More Audio Complaints!

    Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

    Problem Hi all,    While many today have fast Internet connection, there are many still who do not, or are connecting through devices with a slower connect, so light web pages and fast load times are still popular.    If your ASP.NET page …
    How to fix error ""Failed to validate the vCentre certificate. Either install or verify the certificate by using the vSphere Data Protection Configuration utility" when you are trying to connect to VDP instance from Vcenter.
    This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…
    This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…

    884 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now