Solved

SQL Distinct List for fields in very large tables

Posted on 2014-11-02
4
139 Views
Last Modified: 2014-11-03
Hi

I need to create a distinct list of values for columns in tables that are between 5 and 10 million records.
These lists can be up to 200,000 records long. At the moment I am creating them while my app runs and
it takes a long time. How do I get around this? Should I create views or tables that have a distinct list?
0
Comment
Question by:murbro
4 Comments
 
LVL 48

Accepted Solution

by:
PortletPaul earned 250 total points
ID: 40418824
There is no detail information to work with, so a generic answer would be:

Views will probably be faster than what you do now; Tables would be faster than views.

Indexes on the source table will have a big influence on performance.
0
 
LVL 10

Expert Comment

by:HuaMinChen
ID: 40418866
Hi,
1. put "distinct" to the statement
2. You can then select the records into other table, on which there is table constraint that is ensuring uniqueness of all records.
0
 
LVL 9

Assisted Solution

by:Valliappan AN
Valliappan AN earned 250 total points
ID: 40418920
You can use (NOLOCK) attribute to your select query something like this, to avoid locking of all those records while fetching, which will dramatically give good performance. Something like this:

Select DISTINCT field1, field2,...,fieldn FROM [yourtable] (NOLOCK)
JOIN [anothertable] (NOLOCK) ON <join condition>
 
HTH.
0
 

Author Closing Comment

by:murbro
ID: 40419367
Thanks
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Nowadays, some of developer are too much worried about data. Who is using data, who is updating it etc. etc. Because, data is more costlier in term of money and information. So security of data is focusing concern in days. Lets' understand the Au…
Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Viewers will learn how the fundamental information of how to create a table.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now