Solved

SBS2011 Server-based program running EXTREMELY slow on Clients

Posted on 2014-01-27
29
1,887 Views
Last Modified: 2014-01-29
I just set up a new network installation on the weekend that included a Server running SBS2011 and several workstations running Windows 7 Pro 64bit.

Each system appears to run very fast - when running its own software.

However, the clients have one program that is based on the Server.

When they run that particular program, it runs extremely slowly.

On the Server, the program takes exactly 1 minute to load (from first click).

On the Client Workstations, the program takes 4 minutes and 20 seconds to load (on the new machines) and over 10 minutes on an older PC.

On the clients, once the program icon has been clicked on, it cannot be closed or stopped (via the task manager). The only way to stop it is to reboot the client.

I thought that it may be due to the antivirus (Symantec EndPoint Security). However, I found that the load times were not affected in any way - whether or not the antivirus was enabled.

I could really use some help, as the system should be much faster than this, yet it's performing like an old 8086.

The Server that is running SBS2011 is a new Xeon system with 16Gb of memory and 2 x SSD in RAID1. By itself, it runs really fast.

The newest workstations have high-end i7 CPUs, 16Gb RAM, SSD, etc. They run really fast when using Office or other local disk programs. However, when they run the program directly from the shared folder on the Server, the result is a computer running at a snails pace.

Thank you for your help.
0
Comment
Question by:Wiltshire
  • 13
  • 10
  • 4
  • +2
29 Comments
 
LVL 56

Expert Comment

by:Cliff Galiher
Comment Utility
Was this program designed to be run from a server?

Honestly, what you describe is what I'd expect with a client/non-network program that is being run from a file-share or mapped drive so the program has no idea it isn't being run locally. Data heavy programs, like those build on Microsoft Access, don't manage their disk writes or their network usage *at all* because they don't realize they are being run over a network so they can saturate one or both. They do a huge data query and then attempt to load the whole thing at once in "local" memory...which since it is running on a mapped drive, means sending the whole darn thing over the wire uncompressed. Plus the overhead of SMB (the protocol used for filesharing, you don't get the full network read/write speeds.)

Fast Ethernet will downright cripple such a program. Even a gigabit network, where the NIC in the client, the server *and* the switch, will suffer because of the SMB overhead. If you are going to run a "local" program from a mapped drive and if that program is data intensive, you really need 10Gb.  And at the cost per port, it would be far better to actually buy (or build) a program that is network-aware.  Even MS Access has support for ODBC drivers so the program runs locally and accesses only the data remotely. And MS SQL (or Oracle, or even MySQL) will only build on that performance increase.

So hopefully that helps you at least understand what you are seeing and why, based on the limited information you provided. And hopefully it gives you a couple of ideas on how to address the bottleneck.
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
I had something like this happen installing Quickbooks on a network, turned out the firewall on the server needed to have an incoming rule set for the program set in the firewall on the server. Easiest way to test turn off the firewall on the server.
0
 

Author Comment

by:Wiltshire
Comment Utility
I'm using Symantec Endpoint Security Small Business Edition. So far, I've created exclusions for the application and folder. The Windows Firewall appears to be off and the system uses Symantec instead. I'm not too sure if there are Symantec Firewall exclusions too.

Either way, I disabled Symantec to see if it affected the performance and found it didn't.

To answer the first response, the program is indeed designed for networks. It is the largest program in the specific industry and other sites seem to be fine.
0
 
LVL 22

Accepted Solution

by:
Olaf De Ceuster earned 500 total points
Comment Utility
Cliffs explanation is spot on.
You can get a  small performance increase by turning off SMB using GP

and a few network card tweaks on Server Nic:
Run from an admin prompt.
netsh int ip set global taskoffload=disabled
netsh int tcp set global congestion=none
netsh int tcp set global autotuning=disabled
netsh int tcp set global rss=disabled

Or set up an RDP server and keep the data local to the program.

Hope that helps,
Olaf
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
make sure all workstations have the same path to the application data, for example Quickbooks users may have the Q: drive mapped however the app may install to QB data using \\servername\Quickbooksdata. The software will explain which one to use.

I've seen where workstations will run slow if they are all not connected to the data the same way, either through the shared mapped drive, like in the example above Q: or the \\server\data path.

All workstations should access the data exactly the same way.
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
Go to Start / Administrative Tools / Share and Storage Management / Manage Open Files check to see how many connections are open to the app, some workstations may have issues reading the application license files on the server and can create multiple connections which is a sign the app is installed incorrectly either on the workstation or server.

Have you run repair (if there's an option) from Add / Remove Programs.
0
 

Author Comment

by:Wiltshire
Comment Utility
Thank you for the suggestions so far.

When using the program in a workstation, it takes almost a minute to print to a network laser, whereas local programs like Word only take a second to print. I'm assuming that the program is sending data back and forward across the network to the Server.

I also wanted to test the connection to see if may be a hardware issue. I swapped out the switch and it made no difference, but was wondering if it was worth doing a direct connection between a workstation and the Server, therefore excluding the rest of the network and router. It's been ages since the last time I did that and I needed to use a cross-over cable. Do current NICs still require a cross-over cable in a situation like this? It may be a total waste of time, but I'm running out of ideas.

Also, I was wondering if it is at all possible that it may be a problem with DNS or other protocol? I set up the Server as the DHCP Server (although that isn't the default for SBS2011) could it have anything to do with that?
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
simple DNS test, does dnslookup resolve correctly

run ipconfig /flushdns

check for duplicate entries in DNS and DHCP

I wouldn't remove the switch for testing, just my thoughts. It's likely something else.
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
run dcdaig /test:dns on the server and post the results
0
 

Author Comment

by:Wiltshire
Comment Utility
... another afterthought

I was wondering if one of SBS2011s defaults may be causing the issue.

Could something like disk quota (although I'm pretty sure none are set), or some other SBS2011 preset make an impact on programs that run via shares?
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
Could something like disk quota (although I'm pretty sure none are set), or some other SBS2011 preset make an impact on programs that run via shares?
unlikely.

How much free space do you have?

What process is taking up the most resources?
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
from an elevated command prompt run dcdiag /test:dns and post the results.
0
 

Author Comment

by:Wiltshire
Comment Utility
Hi WORKS2011. I'll be doing onsite soon and will run dcdiag /test:dns. Do you want me to run it on the Server, or a workstation, or both?

Also, I just got a call back from the applications tech support team. They suggested changing the DNS to 8.8.8.8 and 8.8.4.4. Aren't they Google addresses? What difference would that make? I currently have DNS pointing to the Server IP.
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
Hi WORKS2011. I'll be doing onsite soon and will run dcdiag /test:dns. Do you want me to run it on the Server, or a workstation, or both?
on the server

Also, I just got a call back from the applications tech support team. They suggested changing the DNS to 8.8.8.8 and 8.8.4.4. Aren't they Google addresses?
yes Google addresses

What difference would that make? I currently have DNS pointing to the Server IP.
Let's wait until you run dcdiag tests and confirm DNS is healthy then we'll look at how DNS is communicating with your ISP DNS.

Good DNS is alright as a workaround but shouldn't need it if DNS is working correctly on the server and communicating correctly with you ISP.
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
Does it take awhile to logon to workstations? This can be an indication DNS is not setup correctly.
0
 
LVL 17

Expert Comment

by:WORKS2011
Comment Utility
Good DNS is alright as a workaround but shouldn't need it if DNS is working correctly on the server and communicating correctly with you ISP.

I meant Google not Good
0
 

Author Comment

by:Wiltshire
Comment Utility
WORKS2011. Logon to to all workstation is fast.
0
 

Author Comment

by:Wiltshire
Comment Utility
Results of dcdiag /test:dns

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\System32>dcdiag /test:dns

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = SERVER
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\SERVER
      Starting test: Connectivity
         ......................... SERVER passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\SERVER

      Starting test: DNS

         DNS Tests are running and not hung. Please wait a few minutes...
         ......................... SERVER passed test DNS

   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : ourdomain

   Running enterprise tests on : ourdomain.local
      Starting test: DNS
         ......................... ourdomain.local passed test DNS
0
 

Author Comment

by:Wiltshire
Comment Utility
How much free space do you have?

322Gb Free of 446Gb Total



What process is taking up the most resources?


Heres a few at the top:

store.exe (SYSTEM) 916616K
sqlservr.exe (NETWORK) 869220K
w3wp.exe (SYSTEM) 779380K
w3wp.exe (NETWORK) 542936K
w3wp.exe (spwebapp) 485236K
sqlservr.exe (SYSTEM) 378016K
DNS.exe (SYSTEM) 309472
w3wp.exe (spfarm) 258804
SemSvc.exe *32 (SYSTEM) 221916K


The process that the application uses is:

nxServer.Exe (nexus) 30,816K
0
 

Author Comment

by:Wiltshire
Comment Utility
There are also a lot of errors in the Event Viewer, such as:



(The Event Viewer is full of this one all day long. The workstation is on and seems to be working okay):

DCOM was unable to communicate with the computer IVUE-9339-02.ourdomain.local using any of the configured protocols.




Also, lots of Kernel-EventTracing errors:

Session "WBCommandletInBuiltTracing" failed to start with the following error: 0xC0000035





Lots of Exchange CmdletLogs errors:

Cmdlet failed. Cmdlet Get-User, parameters {Identity=NT AUTHORITY\SYSTEM}.




Here's another weird error that occurred throughout the night:

DistributedCOM

The machine-default permission settings do not grant Local Activation permission for the COM Server application with CLSID
{000C101C-0000-0000-C000-000000000046}
 and APPID
{000C101C-0000-0000-C000-000000000046}
 to the user OURDOMAIN\spfarm SID (S-1-5-21-4036107107-2153396064-2880781225-1157) from address LocalHost (Using LRPC). This security permission can be modified using the Component Services administrative tool.



Also, a few VSS Warnings regularly:

Volume Shadow Copy Service error: Failed resolving account spsearch with status 1376. Check connection to domain controller and VssAccessControl registry key.

Operation:
   Initializing Writer

Context:
   Writer Class Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Name: COM+ REGDB Writer

Error-specific details:
   Error: NetLocalGroupGetMemebers(spsearch), 0x80070560, The specified local group does not exist.



Hopefully, this will give you a few hints.
0
 

Author Comment

by:Wiltshire
Comment Utility
Out of personal curiosity, I copied 754Mb (791,000Kb) of files from the Server shared drive onto the workstation. It took 30 seconds. I think that this was a reasonable speed for the task, so the fault doesn't appear to be related to transfer speeds.
0
 
LVL 22

Expert Comment

by:David Atkin
Comment Utility
Hello,

I don't know if I've missed this but - Whats the name of the software installed?
0
 

Author Comment

by:Wiltshire
Comment Utility
The name of the software is "Optomate". It is the industry standard for Optometry practices in Australia.

The software was running fine on the previous network, which consisted of a Server running SBS2003 and clients running Windows XP Pro.

The only difference is the new hardware and new Server and workstation OS.

The Optomate program and data remain the same, but for some unknown reason, the new network is unable to run it quickly.

Printing a receipt in Optomate takes 2 minutes. You can imagine how customers feel about waiting that long when all they want to do is go.
0
 
LVL 22

Expert Comment

by:David Atkin
Comment Utility
Do you have any support with Octomate?

Have you installed the software inside the program files? I've had issues previously with certain pieces of old software having issues with the Virtual Store when being installed inside the Program files on Server 2008 64-Bit.

Have you installed the software onto a workstation with a local database to see if the problem still occurs?
0
 

Author Comment

by:Wiltshire
Comment Utility
The software is installed and configured by the Optomate tech support team. They are very good at their task, but their expertise is limited to their software. None of them appear to know much about the underlying Server or workstation operating systems. They are very helpful people, but their knowledge only does so far.
0
 
LVL 22

Expert Comment

by:David Atkin
Comment Utility
It would be worth asking them if they've had any issues on Server 2008 R2 if you haven't already.

Have you installed it onto a Windows XP machine to see if there is still an issue?

If it runs of a Database then it may also be worth asking Optomate to see if there is a process to compact/repair it.

The DNS on your client PCs want to be pointed to your server not googles.  Changing them to googles will cause you issues with local lookups.
0
 

Author Comment

by:Wiltshire
Comment Utility
I didn't see any point in changing the DNS either.

Optomate is used on systems running 2008 R2 all around the country without any problems.

There is an XP workstation currently on the network because it runs a special Eye camera and we couldn't upgrade it to 7 Pro. The XP machine is a lot older and slower. It takes 10 minutes to load Optomate now. On the old network, it only took a few minutes. However, that may purely be because of the current problem we're trying to solve.
0
 
LVL 22

Expert Comment

by:David Atkin
Comment Utility
Yes I imagine that the XP delay is also caused by this problem.  

Did you try the SMB solution offered to you earlier by Olaf?

Would it be possible to get the application installed on another PC using the database locally?  - Just to see if the problem still occurs?

Obviously keep using the software as it is, just do it for one PC.
0
 

Author Closing Comment

by:Wiltshire
Comment Utility
Problem solved !!!

I did several things at the one time, so I’m not sure which made the big difference. I guess it doesn’t matter too much, because it doesn’t take very long to do all three of the suggested fixes anyway:

(1)

Set
DNS1 to Server IP
DNS2 to Router IP

and

(2)

Instead of disabling SMB using GP, I did it by editing the Registry:

Run Regedit.exe

Set all of the following from 1 to 0:

On the Server:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters\Enablesecuritysignature
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters\Requiresecuritysignature

On the Workstation:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanworkstation\Parameters\Enablesecuritysignature
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanworkstation\Parameters\Requiresecuritysignature

and

(3)

A few network card tweaks on Server Nic:

Run from an admin prompt.

netsh int ip set global taskoffload=disabled
netsh int tcp set global congestion=none
netsh int tcp set global autotuning=disabled
netsh int tcp set global rss=disabled



After all of that, I rebooted the Server and Workstations and found that they were now running the program at full speed. The older Notebook client that used to take 10 minutes to open the program (if at all) now takes 2 minutes). The new workstation used to take 4 minutes to load the program, whereas now it only takes 25 seconds to load. That’s a very big improvement, considering that the tech support team told me that the average time it takes to open the program on a new PC is about a minute and a half.

I hope this helps anyone facing similar problems.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

If you are a user of the discontinued Microsoft Office Accounting 2008 (MSOA) and have to move to a new computer running Windows 8, you will be unhappy to discover that it won't install.  In particular, Microsoft SQL Server 2005 Express Edition (SSE…
New Windows 7 Installations take days for Windows-Updates to show up and install. This can easily be fixed. I have finally decided to write an article because this seems to get asked several times a day lately. This Article and the Links apply to…
This tutorial will walk an individual through the steps necessary to configure their installation of BackupExec 2012 to use network shared disk space. Verify that the path to the shared storage is valid and that data can be written to that location:…
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now