Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Pros and Cons of setting up a batch processing server?

Posted on 2016-09-21
8
93 Views
Last Modified: 2016-09-27
We are converting our in-house client/server applications to be web applications using Visual Studio 2012, ASP.net, SQL Server 2014.  We are starting to create some batch jobs that we would like to run without user interaction and one of my team members suggested setting up a batch processing server to handle these processes.  Since this is all new to me, I was wondering if anyone could provide feedback on: the pros and cons of using a batch processing server,
is it really needed,
without it can large data files (over a gig of data) be processed without timing out if there isn't a separate server?

We are a small state agency and most of the batch processes that will be needed are annual processes with a few daily and weekly processes thrown in.

Any information you can provide is greatly appreciated.
Thank You, Ang
0
Comment
Question by:AGenAdmin
  • 4
  • 4
8 Comments
 
LVL 33

Accepted Solution

by:
ste5an earned 500 total points
ID: 41808877
The pros and cons are the same as for any dedicated server: Do you need this much CPU and RAM and disk capacity for it?

But from your description:
Check whether you can afford the license fees for the OS. Then as a small agency start with on server and run your services (SQL, IIS, Batch) in three Hyper-V machines. This will give you the necessary control over each machine (security, privilege separation). And when you really need more performance, then you can either upgrade this machine or buy a second one and move the batch VM on it.

A gigabyte of data is not a problem.

The only important thing: When your batch means ETL database processing, then it may be important to run it in a separate SQL Server service, this can be on the same (virtual) machine or another one. Cause ETL database processing has a different data access pattern. When running ETL processes in the same SQL Server service (process), then it may happen that this leads to performance degradation of your production OLTP services due to the kind of buffer pool handling.
0
 

Author Comment

by:AGenAdmin
ID: 41815947
Ste5an, thanks for your reply.  Like I said I'm new to this type of application environment so what do you mean by ETL database processing?

Thanks,
Ang
0
 
LVL 33

Expert Comment

by:ste5an
ID: 41815959
0
Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

 

Author Comment

by:AGenAdmin
ID: 41815989
Ste5an,
Thanks for the explanation.  We don't have a data warehouse db, all the processes would run against the application dbs.  

I didn't explain that we are running in a Cisco VM environment where we have a production SQL Server virtual  server, a production web server virtual server and asked our network division to set up a batch processing virtual server, but they are questioning if we really need this to be a separate virtual server?  I'm looking for some info to back up our desire to have it be separate. They are trying to preserve our OS licenses and we are trying to prevent performance issues.
Thanks, Ang
0
 
LVL 33

Expert Comment

by:ste5an
ID: 41816075
Okay, then: What kind of batch processing are we talking here about?
0
 

Author Comment

by:AGenAdmin
ID: 41816099
ste5an,
some examples would be updating an employee's org code on various tables when they move from one org to another within the department, running a process that would purge data based on our record retention policy.
0
 
LVL 33

Expert Comment

by:ste5an
ID: 41816105
How would you implement this?

Cause it sounds like a stored procedure could be sufficient. In this case you would simply use a SQL Server Agent job or a scheduled maintenance plan.

Or are we talking about SQL Server Integration Services?
0
 

Author Closing Comment

by:AGenAdmin
ID: 41818011
Thanks for your input, I was able to the information provided to plead my case.
Ang
0

Featured Post

Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Problem Hi all,    While many today have fast Internet connection, there are many still who do not, or are connecting through devices with a slower connect, so light web pages and fast load times are still popular.    If your ASP.NET page …
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
This video shows how to use Hyena, from SystemTools Software, to bulk import 100 user accounts from an external text file. View in 1080p for best video quality.
The Email Laundry PDF encryption service allows companies to send confidential encrypted  emails to anybody. The PDF document can also contain attachments that are embedded in the encrypted PDF. The password is randomly generated by The Email Laundr…

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question