Solved

Pros and Cons of setting up a batch processing server?

Posted on 2016-09-21
8
64 Views
Last Modified: 2016-09-27
We are converting our in-house client/server applications to be web applications using Visual Studio 2012, ASP.net, SQL Server 2014.  We are starting to create some batch jobs that we would like to run without user interaction and one of my team members suggested setting up a batch processing server to handle these processes.  Since this is all new to me, I was wondering if anyone could provide feedback on: the pros and cons of using a batch processing server,
is it really needed,
without it can large data files (over a gig of data) be processed without timing out if there isn't a separate server?

We are a small state agency and most of the batch processes that will be needed are annual processes with a few daily and weekly processes thrown in.

Any information you can provide is greatly appreciated.
Thank You, Ang
0
Comment
Question by:AGenAdmin
  • 4
  • 4
8 Comments
 
LVL 33

Accepted Solution

by:
ste5an earned 500 total points
ID: 41808877
The pros and cons are the same as for any dedicated server: Do you need this much CPU and RAM and disk capacity for it?

But from your description:
Check whether you can afford the license fees for the OS. Then as a small agency start with on server and run your services (SQL, IIS, Batch) in three Hyper-V machines. This will give you the necessary control over each machine (security, privilege separation). And when you really need more performance, then you can either upgrade this machine or buy a second one and move the batch VM on it.

A gigabyte of data is not a problem.

The only important thing: When your batch means ETL database processing, then it may be important to run it in a separate SQL Server service, this can be on the same (virtual) machine or another one. Cause ETL database processing has a different data access pattern. When running ETL processes in the same SQL Server service (process), then it may happen that this leads to performance degradation of your production OLTP services due to the kind of buffer pool handling.
0
 

Author Comment

by:AGenAdmin
ID: 41815947
Ste5an, thanks for your reply.  Like I said I'm new to this type of application environment so what do you mean by ETL database processing?

Thanks,
Ang
0
 
LVL 33

Expert Comment

by:ste5an
ID: 41815959
0
 

Author Comment

by:AGenAdmin
ID: 41815989
Ste5an,
Thanks for the explanation.  We don't have a data warehouse db, all the processes would run against the application dbs.  

I didn't explain that we are running in a Cisco VM environment where we have a production SQL Server virtual  server, a production web server virtual server and asked our network division to set up a batch processing virtual server, but they are questioning if we really need this to be a separate virtual server?  I'm looking for some info to back up our desire to have it be separate. They are trying to preserve our OS licenses and we are trying to prevent performance issues.
Thanks, Ang
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 
LVL 33

Expert Comment

by:ste5an
ID: 41816075
Okay, then: What kind of batch processing are we talking here about?
0
 

Author Comment

by:AGenAdmin
ID: 41816099
ste5an,
some examples would be updating an employee's org code on various tables when they move from one org to another within the department, running a process that would purge data based on our record retention policy.
0
 
LVL 33

Expert Comment

by:ste5an
ID: 41816105
How would you implement this?

Cause it sounds like a stored procedure could be sufficient. In this case you would simply use a SQL Server Agent job or a scheduled maintenance plan.

Or are we talking about SQL Server Integration Services?
0
 

Author Closing Comment

by:AGenAdmin
ID: 41818011
Thanks for your input, I was able to the information provided to plead my case.
Ang
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Problem Hi all,    While many today have fast Internet connection, there are many still who do not, or are connecting through devices with a slower connect, so light web pages and fast load times are still popular.    If your ASP.NET page …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
This Micro Tutorial will teach you how to censor certain areas of your screen. The example in this video will show a little boy's face being blurred. This will be demonstrated using Adobe Premiere Pro CS6.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now