Pros and Cons of setting up a batch processing server?

We are converting our in-house client/server applications to be web applications using Visual Studio 2012, ASP.net, SQL Server 2014.  We are starting to create some batch jobs that we would like to run without user interaction and one of my team members suggested setting up a batch processing server to handle these processes.  Since this is all new to me, I was wondering if anyone could provide feedback on: the pros and cons of using a batch processing server,
is it really needed,
without it can large data files (over a gig of data) be processed without timing out if there isn't a separate server?

We are a small state agency and most of the batch processes that will be needed are annual processes with a few daily and weekly processes thrown in.

Any information you can provide is greatly appreciated.
Thank You, Ang
AGenAdminAsked:
Who is Participating?
 
ste5anConnect With a Mentor Senior DeveloperCommented:
The pros and cons are the same as for any dedicated server: Do you need this much CPU and RAM and disk capacity for it?

But from your description:
Check whether you can afford the license fees for the OS. Then as a small agency start with on server and run your services (SQL, IIS, Batch) in three Hyper-V machines. This will give you the necessary control over each machine (security, privilege separation). And when you really need more performance, then you can either upgrade this machine or buy a second one and move the batch VM on it.

A gigabyte of data is not a problem.

The only important thing: When your batch means ETL database processing, then it may be important to run it in a separate SQL Server service, this can be on the same (virtual) machine or another one. Cause ETL database processing has a different data access pattern. When running ETL processes in the same SQL Server service (process), then it may happen that this leads to performance degradation of your production OLTP services due to the kind of buffer pool handling.
0
 
AGenAdminAuthor Commented:
Ste5an, thanks for your reply.  Like I said I'm new to this type of application environment so what do you mean by ETL database processing?

Thanks,
Ang
0
 
ste5anSenior DeveloperCommented:
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
AGenAdminAuthor Commented:
Ste5an,
Thanks for the explanation.  We don't have a data warehouse db, all the processes would run against the application dbs.  

I didn't explain that we are running in a Cisco VM environment where we have a production SQL Server virtual  server, a production web server virtual server and asked our network division to set up a batch processing virtual server, but they are questioning if we really need this to be a separate virtual server?  I'm looking for some info to back up our desire to have it be separate. They are trying to preserve our OS licenses and we are trying to prevent performance issues.
Thanks, Ang
0
 
ste5anSenior DeveloperCommented:
Okay, then: What kind of batch processing are we talking here about?
0
 
AGenAdminAuthor Commented:
ste5an,
some examples would be updating an employee's org code on various tables when they move from one org to another within the department, running a process that would purge data based on our record retention policy.
0
 
ste5anSenior DeveloperCommented:
How would you implement this?

Cause it sounds like a stored procedure could be sufficient. In this case you would simply use a SQL Server Agent job or a scheduled maintenance plan.

Or are we talking about SQL Server Integration Services?
0
 
AGenAdminAuthor Commented:
Thanks for your input, I was able to the information provided to plead my case.
Ang
0
All Courses

From novice to tech pro — start learning today.