SolvedPrivate

SSIS Data Load

Posted on 2013-05-17
3
27 Views
Last Modified: 2016-02-10
Hi Experts,

I have attached a query below. It is returning the results I want perfectly. Now I will have to load the results starting from '2009-01-01 00:00:00.000'  until '2013-05-16 00:00:00.000'. I will have to load the daily records stating from '2009-01-01 00:00:00.000' period into my table i.e

 where
rec_create_date between convert(varchar,'2009-1-01 00:00:00.000',102)
and convert(varchar,'2009-01-02 00:00:00.000',102)

and again
where
rec_create_date between convert(varchar,'2009-1-02 00:00:00.000',102)
and convert(varchar,'2009-01-03 00:00:00.000',102)

so on and so forth. These records are coming from Prod into my Rep Server. It is very tough to change the date everytime I run the package on the where condition. Could it be done any better so that I don't have to change the date in the source at all and it will itself change the where condition. I know it can be done with the help of a variable but can you explain me in detail of how to set up this step in my package. It would be a great help. Thanks.
Qry1.sql
0
Comment
Question by:Josh2442
  • 2
3 Comments
 
LVL 48

Accepted Solution

by:
PortletPaul earned 500 total points
Comment Utility
I'm going to offer an observation here: do NOT use "between"

as is:

rec_create_date between convert(varchar,'2009-1-01 00:00:00.000',102)
and convert(varchar,'2009-01-02 00:00:00.000',102)

to be:
(
rec_create_date >= convert(varchar,'2009-1-01 00:00:00.000',102)
and rec_create_date < convert(varchar,'2009-01-02 00:00:00.000',102)
)

as you are performing an import, you run the (small) risk of records being equal to a time of 00:00:00.000, and because you are using "between" then there is a (small) risk of duplication

The safest, most reliable, method is a combination of >= with < as indicated above.
(don't forget the parentheses)

You might regard the risk to low to bother, that's fine, but please just note that using "between" isn't best practice for date range filters.

for more on this topic please see: "Beware of Between"
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
To your specific question about variables in SSIS, I suggest you try these:

https://www.simple-talk.com/sql/ssis/working-with-variables-in-sql-server-integration-services/

http://dataqueen.unlimitedviz.com/2012/08/how-to-set-and-use-variables-in-ssis-execute-sql-task/

These should answer everything you would need to implement variables in that import I believe.
0
 

Author Closing Comment

by:Josh2442
Comment Utility
Thanks Expert
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Suggested Solutions

by Mark Wills Attending one of Rob Farley's seminars the other day, I heard the phrase "The Accidental DBA" and fell in love with it. It got me thinking about the plight of the newcomer to SQL Server...  So if you are the accidental DBA, or, simp…
Slowly Changing Dimension Transformation component in data task flow is very useful for us to manage and control how data changes in SSIS.
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

6 Experts available now in Live!

Get 1:1 Help Now