Running ETL siss in virtual machine


Having issue with slow processing in virtual environment as opposed to physical server?

As anyone come a cross situation where running large etl packages on a virtual machine?
the Ssis packages uses SCD component.
My package is really slow in term running on virtual environment rather than physical environment.

Any one explain to me why this.

ken hanseAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

JohnBusiness Consultant (Owner)Commented:
For what you want to be fast, you need to use ESX so that you have the fastest possible hardware server, have fast disks in the machine (possibly even commercial, enterprise SSD drives), and lots of memory. A virtual machine mounted in such an environment should process nearly as quickly as a real server.
ken hanseAuthor Commented:
thanks John,

My ETLs runs two days in virtual environment compare to the production physical server 3hrs.

my infrastructure guy told me 8CPU in a virtual machine said good capacity on virtual server. Trust me two days to process same thing running on 3hrs in a physical environment.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
JohnBusiness Consultant (Owner)Commented:
You need a fast hardware architecture as I noted to speed things up. 2 days to do 3 hours work means the hardware / virtual environment is entirely too slow.

It is not the same thing, but I can load up a Windows virtual machine on my ThinkPad and once running, it will process updates, spreadsheets and documents as fast as the host machine.
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

Vitor MontalvãoMSSQL Senior EngineerCommented:
my infrastructure guy told me 8CPU in a virtual machine said good capacity on virtual server
I don't agree totally with that. Don't forget that they are vCPU and not CPU and are shared with other virtual machines. I would reduce it to half (4 vCPU) and increase the memory. SSIS packages can consume a lot of memory.
David Johnson, CD, MVPOwnerCommented:
you really have to compare apples to apples here. The biggest item you have to consider is where are the bottlenecks.. disk i/o is the normal #1 problem. #2 is RAM is there enough free ram available and lastly cpu performance.  For instance I split a sql database file locations from the default system drive where sql competed with everything else for iops and put the database / log / and temp files each on a separate spindle (physical drive) and took a 3 hour operation down to 5 minutes. Personally I'm heavily invested in Storage Spaces and its storage tiering which puts the most used data onto SSD's either automatically based on demand or manual placing the data on SSD's
ken hanseAuthor Commented:
really good feed back
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server 2008

From novice to tech pro — start learning today.