Solved

I need to download data from a huge table month wise

Posted on 2014-10-13
6
129 Views
Last Modified: 2016-02-11
Hi Experts,
I need to query a table from production which is huge for the past 6 months, I was suggested to query the prod table by month and load into 6 dev tables, not sure how I can do that?
Would SSIS be a good option? please suggest?
0
Comment
Question by:sqlcurious
6 Comments
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 40377901
the request is quite unclear/unprecise...

the question is, what exactly you have as data and what you need as results...

one option might be to partition the (original) table, to start with:
http://msdn.microsoft.com/en-us/library/ms190787.aspx
0
 
LVL 69

Expert Comment

by:ScottPletcher
ID: 40378104
What is the clustering index on the original table?  If it is month/date[time], as it seems likely it should be, then querying by month makes sense.

I wouldn't load into 6 separate tables in dev.  Instead, just be sure to cluster the table by month/date[time].
0
 
LVL 13

Expert Comment

by:Russell Fox
ID: 40378132
See above for long-term solutions, but if this is a one-off project then this will create the new tables as it pulls data from the existing table:
SELECT * 
INTO Production201401
FROM dbo.Production
WHERE CAST([OrderDate] AS DATE) BETWEEN '1/1/2014' AND '1/31/2014'
GO
SELECT * 
INTO Production201402
FROM dbo.Production
WHERE CAST([OrderDate] AS DATE) BETWEEN '2/1/2014' AND '2/28/2014'
GO
-- Etc.

Open in new window

0
Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

 
LVL 69

Accepted Solution

by:
ScottPletcher earned 500 total points
ID: 40378143
No!  You never  use a cast a table column or use a function on one [if it can be avoided], as that disallows any index seek.  On huge tables that could be a major performance  issue, esp. when you run the command six times!

Also, literal date should always be coded as 'YYYYMMDD', as that format always works, regardless of SQL settings.

Finally, the best way to check dates/datetimes is with >= and <, not with between.

For example:

WHERE OrderDate >= '20140101' AND OrderDate < '20140201'
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 40381412
Adding for emphasis, do not use BETWEEN for date ranges:

... the best practice with date and time ranges is to avoid BETWEEN and to always use the form:

WHERE col >= '20120101' AND col < '20120201'
This form works with all types and all precisions, regardless of whether the time part is applicable.
Itzik Ben-Gan

This applies to many databases using SQL. Also see: "Beware of Between"
0
 

Author Closing Comment

by:sqlcurious
ID: 40436356
Thanks
0

Featured Post

Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A couple of weeks ago, my client requested me to implement a SSIS package that allows them to download their files from a FTP server and archives them. Microsoft SSIS is the powerful tool which allows us to proceed multiple files at same time even w…
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.
Via a live example, show how to set up a backup for SQL Server using a Maintenance Plan and how to schedule the job into SQL Server Agent.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now