Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Pros and Cons of setting up a batch processing server?

Posted on 2016-09-21
8
Medium Priority
?
207 Views
Last Modified: 2016-09-27
We are converting our in-house client/server applications to be web applications using Visual Studio 2012, ASP.net, SQL Server 2014.  We are starting to create some batch jobs that we would like to run without user interaction and one of my team members suggested setting up a batch processing server to handle these processes.  Since this is all new to me, I was wondering if anyone could provide feedback on: the pros and cons of using a batch processing server,
is it really needed,
without it can large data files (over a gig of data) be processed without timing out if there isn't a separate server?

We are a small state agency and most of the batch processes that will be needed are annual processes with a few daily and weekly processes thrown in.

Any information you can provide is greatly appreciated.
Thank You, Ang
0
Comment
Question by:AGenAdmin
  • 4
  • 4
8 Comments
 
LVL 36

Accepted Solution

by:
ste5an earned 2000 total points
ID: 41808877
The pros and cons are the same as for any dedicated server: Do you need this much CPU and RAM and disk capacity for it?

But from your description:
Check whether you can afford the license fees for the OS. Then as a small agency start with on server and run your services (SQL, IIS, Batch) in three Hyper-V machines. This will give you the necessary control over each machine (security, privilege separation). And when you really need more performance, then you can either upgrade this machine or buy a second one and move the batch VM on it.

A gigabyte of data is not a problem.

The only important thing: When your batch means ETL database processing, then it may be important to run it in a separate SQL Server service, this can be on the same (virtual) machine or another one. Cause ETL database processing has a different data access pattern. When running ETL processes in the same SQL Server service (process), then it may happen that this leads to performance degradation of your production OLTP services due to the kind of buffer pool handling.
0
 

Author Comment

by:AGenAdmin
ID: 41815947
Ste5an, thanks for your reply.  Like I said I'm new to this type of application environment so what do you mean by ETL database processing?

Thanks,
Ang
0
 
LVL 36

Expert Comment

by:ste5an
ID: 41815959
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:AGenAdmin
ID: 41815989
Ste5an,
Thanks for the explanation.  We don't have a data warehouse db, all the processes would run against the application dbs.  

I didn't explain that we are running in a Cisco VM environment where we have a production SQL Server virtual  server, a production web server virtual server and asked our network division to set up a batch processing virtual server, but they are questioning if we really need this to be a separate virtual server?  I'm looking for some info to back up our desire to have it be separate. They are trying to preserve our OS licenses and we are trying to prevent performance issues.
Thanks, Ang
0
 
LVL 36

Expert Comment

by:ste5an
ID: 41816075
Okay, then: What kind of batch processing are we talking here about?
0
 

Author Comment

by:AGenAdmin
ID: 41816099
ste5an,
some examples would be updating an employee's org code on various tables when they move from one org to another within the department, running a process that would purge data based on our record retention policy.
0
 
LVL 36

Expert Comment

by:ste5an
ID: 41816105
How would you implement this?

Cause it sounds like a stored procedure could be sufficient. In this case you would simply use a SQL Server Agent job or a scheduled maintenance plan.

Or are we talking about SQL Server Integration Services?
0
 

Author Closing Comment

by:AGenAdmin
ID: 41818011
Thanks for your input, I was able to the information provided to plead my case.
Ang
0

Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

IntroductionWhile developing web applications, a single page might contain many regions and each region might contain many number of controls with the capability to perform  postback. Many times you might need to perform some action on an ASP.NET po…
VALIDATING DATES One method of validating dates is to jam the date into the DATE command and see if it accepts it by examining the system's errorlevel value. A non-zero result indicates failure. A typical example might look something like the fol…
This Micro Tutorial will teach you how to add a cinematic look to any film or video out there. There are very few simple steps that you will follow to do so. This will be demonstrated using Adobe Premiere Pro CS6.
In response to a need for security and privacy, and to continue fostering an environment members can turn to for support, solutions, and education, Experts Exchange has created anonymous question capabilities. This new feature is available to our Pr…
Suggested Courses

926 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question