Solved

Pros and Cons of setting up a batch processing server?

Posted on 2016-09-21
8
54 Views
Last Modified: 2016-09-27
We are converting our in-house client/server applications to be web applications using Visual Studio 2012, ASP.net, SQL Server 2014.  We are starting to create some batch jobs that we would like to run without user interaction and one of my team members suggested setting up a batch processing server to handle these processes.  Since this is all new to me, I was wondering if anyone could provide feedback on: the pros and cons of using a batch processing server,
is it really needed,
without it can large data files (over a gig of data) be processed without timing out if there isn't a separate server?

We are a small state agency and most of the batch processes that will be needed are annual processes with a few daily and weekly processes thrown in.

Any information you can provide is greatly appreciated.
Thank You, Ang
0
Comment
Question by:AGenAdmin
  • 4
  • 4
8 Comments
 
LVL 32

Accepted Solution

by:
ste5an earned 500 total points
ID: 41808877
The pros and cons are the same as for any dedicated server: Do you need this much CPU and RAM and disk capacity for it?

But from your description:
Check whether you can afford the license fees for the OS. Then as a small agency start with on server and run your services (SQL, IIS, Batch) in three Hyper-V machines. This will give you the necessary control over each machine (security, privilege separation). And when you really need more performance, then you can either upgrade this machine or buy a second one and move the batch VM on it.

A gigabyte of data is not a problem.

The only important thing: When your batch means ETL database processing, then it may be important to run it in a separate SQL Server service, this can be on the same (virtual) machine or another one. Cause ETL database processing has a different data access pattern. When running ETL processes in the same SQL Server service (process), then it may happen that this leads to performance degradation of your production OLTP services due to the kind of buffer pool handling.
0
 

Author Comment

by:AGenAdmin
ID: 41815947
Ste5an, thanks for your reply.  Like I said I'm new to this type of application environment so what do you mean by ETL database processing?

Thanks,
Ang
0
 
LVL 32

Expert Comment

by:ste5an
ID: 41815959
0
 

Author Comment

by:AGenAdmin
ID: 41815989
Ste5an,
Thanks for the explanation.  We don't have a data warehouse db, all the processes would run against the application dbs.  

I didn't explain that we are running in a Cisco VM environment where we have a production SQL Server virtual  server, a production web server virtual server and asked our network division to set up a batch processing virtual server, but they are questioning if we really need this to be a separate virtual server?  I'm looking for some info to back up our desire to have it be separate. They are trying to preserve our OS licenses and we are trying to prevent performance issues.
Thanks, Ang
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 32

Expert Comment

by:ste5an
ID: 41816075
Okay, then: What kind of batch processing are we talking here about?
0
 

Author Comment

by:AGenAdmin
ID: 41816099
ste5an,
some examples would be updating an employee's org code on various tables when they move from one org to another within the department, running a process that would purge data based on our record retention policy.
0
 
LVL 32

Expert Comment

by:ste5an
ID: 41816105
How would you implement this?

Cause it sounds like a stored procedure could be sufficient. In this case you would simply use a SQL Server Agent job or a scheduled maintenance plan.

Or are we talking about SQL Server Integration Services?
0
 

Author Closing Comment

by:AGenAdmin
ID: 41818011
Thanks for your input, I was able to the information provided to plead my case.
Ang
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Robocopy command line 6 34
network copying issue 4 40
how to find then copy to anotehr file 3 26
Paging GridView 7 34
If like me you are one who spends a lot of time working and scripting with cmd.exe, sometimes it is handy to be able to quickly view a calendar for a given month and year. This script will quickly do just that!  Save the code posted below to a .bat …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now