Solved

SBS2011 Server-based program running EXTREMELY slow on Clients

Posted on 2014-01-27
29
2,106 Views
Last Modified: 2014-01-29
I just set up a new network installation on the weekend that included a Server running SBS2011 and several workstations running Windows 7 Pro 64bit.

Each system appears to run very fast - when running its own software.

However, the clients have one program that is based on the Server.

When they run that particular program, it runs extremely slowly.

On the Server, the program takes exactly 1 minute to load (from first click).

On the Client Workstations, the program takes 4 minutes and 20 seconds to load (on the new machines) and over 10 minutes on an older PC.

On the clients, once the program icon has been clicked on, it cannot be closed or stopped (via the task manager). The only way to stop it is to reboot the client.

I thought that it may be due to the antivirus (Symantec EndPoint Security). However, I found that the load times were not affected in any way - whether or not the antivirus was enabled.

I could really use some help, as the system should be much faster than this, yet it's performing like an old 8086.

The Server that is running SBS2011 is a new Xeon system with 16Gb of memory and 2 x SSD in RAID1. By itself, it runs really fast.

The newest workstations have high-end i7 CPUs, 16Gb RAM, SSD, etc. They run really fast when using Office or other local disk programs. However, when they run the program directly from the shared folder on the Server, the result is a computer running at a snails pace.

Thank you for your help.
0
Comment
Question by:Wiltshire
  • 13
  • 10
  • 4
  • +2
29 Comments
 
LVL 57

Expert Comment

by:Cliff Galiher
ID: 39814092
Was this program designed to be run from a server?

Honestly, what you describe is what I'd expect with a client/non-network program that is being run from a file-share or mapped drive so the program has no idea it isn't being run locally. Data heavy programs, like those build on Microsoft Access, don't manage their disk writes or their network usage *at all* because they don't realize they are being run over a network so they can saturate one or both. They do a huge data query and then attempt to load the whole thing at once in "local" memory...which since it is running on a mapped drive, means sending the whole darn thing over the wire uncompressed. Plus the overhead of SMB (the protocol used for filesharing, you don't get the full network read/write speeds.)

Fast Ethernet will downright cripple such a program. Even a gigabit network, where the NIC in the client, the server *and* the switch, will suffer because of the SMB overhead. If you are going to run a "local" program from a mapped drive and if that program is data intensive, you really need 10Gb.  And at the cost per port, it would be far better to actually buy (or build) a program that is network-aware.  Even MS Access has support for ODBC drivers so the program runs locally and accesses only the data remotely. And MS SQL (or Oracle, or even MySQL) will only build on that performance increase.

So hopefully that helps you at least understand what you are seeing and why, based on the limited information you provided. And hopefully it gives you a couple of ideas on how to address the bottleneck.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39814108
I had something like this happen installing Quickbooks on a network, turned out the firewall on the server needed to have an incoming rule set for the program set in the firewall on the server. Easiest way to test turn off the firewall on the server.
0
 

Author Comment

by:Wiltshire
ID: 39814125
I'm using Symantec Endpoint Security Small Business Edition. So far, I've created exclusions for the application and folder. The Windows Firewall appears to be off and the system uses Symantec instead. I'm not too sure if there are Symantec Firewall exclusions too.

Either way, I disabled Symantec to see if it affected the performance and found it didn't.

To answer the first response, the program is indeed designed for networks. It is the largest program in the specific industry and other sites seem to be fine.
0
Are your AD admin tools letting you down?

Managing Active Directory can get complicated.  Often, the native tools for managing AD are just not up to the task.  The largest Active Directory installations in the world have relied on one tool to manage their day-to-day administration tasks: Hyena. Start your trial today.

 
LVL 22

Accepted Solution

by:
Olaf De Ceuster earned 500 total points
ID: 39814152
Cliffs explanation is spot on.
You can get a  small performance increase by turning off SMB using GP

and a few network card tweaks on Server Nic:
Run from an admin prompt.
netsh int ip set global taskoffload=disabled
netsh int tcp set global congestion=none
netsh int tcp set global autotuning=disabled
netsh int tcp set global rss=disabled

Or set up an RDP server and keep the data local to the program.

Hope that helps,
Olaf
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39814959
make sure all workstations have the same path to the application data, for example Quickbooks users may have the Q: drive mapped however the app may install to QB data using \\servername\Quickbooksdata. The software will explain which one to use.

I've seen where workstations will run slow if they are all not connected to the data the same way, either through the shared mapped drive, like in the example above Q: or the \\server\data path.

All workstations should access the data exactly the same way.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39814975
Go to Start / Administrative Tools / Share and Storage Management / Manage Open Files check to see how many connections are open to the app, some workstations may have issues reading the application license files on the server and can create multiple connections which is a sign the app is installed incorrectly either on the workstation or server.

Have you run repair (if there's an option) from Add / Remove Programs.
0
 

Author Comment

by:Wiltshire
ID: 39816706
Thank you for the suggestions so far.

When using the program in a workstation, it takes almost a minute to print to a network laser, whereas local programs like Word only take a second to print. I'm assuming that the program is sending data back and forward across the network to the Server.

I also wanted to test the connection to see if may be a hardware issue. I swapped out the switch and it made no difference, but was wondering if it was worth doing a direct connection between a workstation and the Server, therefore excluding the rest of the network and router. It's been ages since the last time I did that and I needed to use a cross-over cable. Do current NICs still require a cross-over cable in a situation like this? It may be a total waste of time, but I'm running out of ideas.

Also, I was wondering if it is at all possible that it may be a problem with DNS or other protocol? I set up the Server as the DHCP Server (although that isn't the default for SBS2011) could it have anything to do with that?
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816713
simple DNS test, does dnslookup resolve correctly

run ipconfig /flushdns

check for duplicate entries in DNS and DHCP

I wouldn't remove the switch for testing, just my thoughts. It's likely something else.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816714
run dcdaig /test:dns on the server and post the results
0
 

Author Comment

by:Wiltshire
ID: 39816728
... another afterthought

I was wondering if one of SBS2011s defaults may be causing the issue.

Could something like disk quota (although I'm pretty sure none are set), or some other SBS2011 preset make an impact on programs that run via shares?
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816734
Could something like disk quota (although I'm pretty sure none are set), or some other SBS2011 preset make an impact on programs that run via shares?
unlikely.

How much free space do you have?

What process is taking up the most resources?
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816735
from an elevated command prompt run dcdiag /test:dns and post the results.
0
 

Author Comment

by:Wiltshire
ID: 39816754
Hi WORKS2011. I'll be doing onsite soon and will run dcdiag /test:dns. Do you want me to run it on the Server, or a workstation, or both?

Also, I just got a call back from the applications tech support team. They suggested changing the DNS to 8.8.8.8 and 8.8.4.4. Aren't they Google addresses? What difference would that make? I currently have DNS pointing to the Server IP.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816759
Hi WORKS2011. I'll be doing onsite soon and will run dcdiag /test:dns. Do you want me to run it on the Server, or a workstation, or both?
on the server

Also, I just got a call back from the applications tech support team. They suggested changing the DNS to 8.8.8.8 and 8.8.4.4. Aren't they Google addresses?
yes Google addresses

What difference would that make? I currently have DNS pointing to the Server IP.
Let's wait until you run dcdiag tests and confirm DNS is healthy then we'll look at how DNS is communicating with your ISP DNS.

Good DNS is alright as a workaround but shouldn't need it if DNS is working correctly on the server and communicating correctly with you ISP.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816760
Does it take awhile to logon to workstations? This can be an indication DNS is not setup correctly.
0
 
LVL 17

Expert Comment

by:WORKS2011
ID: 39816766
Good DNS is alright as a workaround but shouldn't need it if DNS is working correctly on the server and communicating correctly with you ISP.

I meant Google not Good
0
 

Author Comment

by:Wiltshire
ID: 39816776
WORKS2011. Logon to to all workstation is fast.
0
 

Author Comment

by:Wiltshire
ID: 39816852
Results of dcdiag /test:dns

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\System32>dcdiag /test:dns

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = SERVER
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\SERVER
      Starting test: Connectivity
         ......................... SERVER passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\SERVER

      Starting test: DNS

         DNS Tests are running and not hung. Please wait a few minutes...
         ......................... SERVER passed test DNS

   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : ourdomain

   Running enterprise tests on : ourdomain.local
      Starting test: DNS
         ......................... ourdomain.local passed test DNS
0
 

Author Comment

by:Wiltshire
ID: 39816861
How much free space do you have?

322Gb Free of 446Gb Total



What process is taking up the most resources?


Heres a few at the top:

store.exe (SYSTEM) 916616K
sqlservr.exe (NETWORK) 869220K
w3wp.exe (SYSTEM) 779380K
w3wp.exe (NETWORK) 542936K
w3wp.exe (spwebapp) 485236K
sqlservr.exe (SYSTEM) 378016K
DNS.exe (SYSTEM) 309472
w3wp.exe (spfarm) 258804
SemSvc.exe *32 (SYSTEM) 221916K


The process that the application uses is:

nxServer.Exe (nexus) 30,816K
0
 

Author Comment

by:Wiltshire
ID: 39816878
There are also a lot of errors in the Event Viewer, such as:



(The Event Viewer is full of this one all day long. The workstation is on and seems to be working okay):

DCOM was unable to communicate with the computer IVUE-9339-02.ourdomain.local using any of the configured protocols.




Also, lots of Kernel-EventTracing errors:

Session "WBCommandletInBuiltTracing" failed to start with the following error: 0xC0000035





Lots of Exchange CmdletLogs errors:

Cmdlet failed. Cmdlet Get-User, parameters {Identity=NT AUTHORITY\SYSTEM}.




Here's another weird error that occurred throughout the night:

DistributedCOM

The machine-default permission settings do not grant Local Activation permission for the COM Server application with CLSID
{000C101C-0000-0000-C000-000000000046}
 and APPID
{000C101C-0000-0000-C000-000000000046}
 to the user OURDOMAIN\spfarm SID (S-1-5-21-4036107107-2153396064-2880781225-1157) from address LocalHost (Using LRPC). This security permission can be modified using the Component Services administrative tool.



Also, a few VSS Warnings regularly:

Volume Shadow Copy Service error: Failed resolving account spsearch with status 1376. Check connection to domain controller and VssAccessControl registry key.

Operation:
   Initializing Writer

Context:
   Writer Class Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Name: COM+ REGDB Writer

Error-specific details:
   Error: NetLocalGroupGetMemebers(spsearch), 0x80070560, The specified local group does not exist.



Hopefully, this will give you a few hints.
0
 

Author Comment

by:Wiltshire
ID: 39816887
Out of personal curiosity, I copied 754Mb (791,000Kb) of files from the Server shared drive onto the workstation. It took 30 seconds. I think that this was a reasonable speed for the task, so the fault doesn't appear to be related to transfer speeds.
0
 
LVL 22

Expert Comment

by:David Atkin
ID: 39817236
Hello,

I don't know if I've missed this but - Whats the name of the software installed?
0
 

Author Comment

by:Wiltshire
ID: 39817354
The name of the software is "Optomate". It is the industry standard for Optometry practices in Australia.

The software was running fine on the previous network, which consisted of a Server running SBS2003 and clients running Windows XP Pro.

The only difference is the new hardware and new Server and workstation OS.

The Optomate program and data remain the same, but for some unknown reason, the new network is unable to run it quickly.

Printing a receipt in Optomate takes 2 minutes. You can imagine how customers feel about waiting that long when all they want to do is go.
0
 
LVL 22

Expert Comment

by:David Atkin
ID: 39817369
Do you have any support with Octomate?

Have you installed the software inside the program files? I've had issues previously with certain pieces of old software having issues with the Virtual Store when being installed inside the Program files on Server 2008 64-Bit.

Have you installed the software onto a workstation with a local database to see if the problem still occurs?
0
 

Author Comment

by:Wiltshire
ID: 39817415
The software is installed and configured by the Optomate tech support team. They are very good at their task, but their expertise is limited to their software. None of them appear to know much about the underlying Server or workstation operating systems. They are very helpful people, but their knowledge only does so far.
0
 
LVL 22

Expert Comment

by:David Atkin
ID: 39817443
It would be worth asking them if they've had any issues on Server 2008 R2 if you haven't already.

Have you installed it onto a Windows XP machine to see if there is still an issue?

If it runs of a Database then it may also be worth asking Optomate to see if there is a process to compact/repair it.

The DNS on your client PCs want to be pointed to your server not googles.  Changing them to googles will cause you issues with local lookups.
0
 

Author Comment

by:Wiltshire
ID: 39817572
I didn't see any point in changing the DNS either.

Optomate is used on systems running 2008 R2 all around the country without any problems.

There is an XP workstation currently on the network because it runs a special Eye camera and we couldn't upgrade it to 7 Pro. The XP machine is a lot older and slower. It takes 10 minutes to load Optomate now. On the old network, it only took a few minutes. However, that may purely be because of the current problem we're trying to solve.
0
 
LVL 22

Expert Comment

by:David Atkin
ID: 39817695
Yes I imagine that the XP delay is also caused by this problem.  

Did you try the SMB solution offered to you earlier by Olaf?

Would it be possible to get the application installed on another PC using the database locally?  - Just to see if the problem still occurs?

Obviously keep using the software as it is, just do it for one PC.
0
 

Author Closing Comment

by:Wiltshire
ID: 39819651
Problem solved !!!

I did several things at the one time, so I’m not sure which made the big difference. I guess it doesn’t matter too much, because it doesn’t take very long to do all three of the suggested fixes anyway:

(1)

Set
DNS1 to Server IP
DNS2 to Router IP

and

(2)

Instead of disabling SMB using GP, I did it by editing the Registry:

Run Regedit.exe

Set all of the following from 1 to 0:

On the Server:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters\Enablesecuritysignature
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters\Requiresecuritysignature

On the Workstation:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanworkstation\Parameters\Enablesecuritysignature
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanworkstation\Parameters\Requiresecuritysignature

and

(3)

A few network card tweaks on Server Nic:

Run from an admin prompt.

netsh int ip set global taskoffload=disabled
netsh int tcp set global congestion=none
netsh int tcp set global autotuning=disabled
netsh int tcp set global rss=disabled



After all of that, I rebooted the Server and Workstations and found that they were now running the program at full speed. The older Notebook client that used to take 10 minutes to open the program (if at all) now takes 2 minutes). The new workstation used to take 4 minutes to load the program, whereas now it only takes 25 seconds to load. That’s a very big improvement, considering that the tech support team told me that the average time it takes to open the program on a new PC is about a minute and a half.

I hope this helps anyone facing similar problems.
0

Featured Post

NAS Cloud Backup Strategies

This article explains backup scenarios when using network storage. We review the so-called “3-2-1 strategy” and summarize the methods you can use to send NAS data to the cloud

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Problem to setup GUI 11 46
Server 2008 R2 WSUS - Clients not pulling via Group Policy 2 30
NTP problem 24 41
Migrating Roaming Profiles to new server 5 25
I’m often asked about newer and larger USB drives connected to SBS2008 and 2011 failing Windows Server Backup vs the older USB drives not failing. As disk space continues to grow and drive technology change SBS2008 and some SBS2011 end up with the f…
You might have come across a situation when you have Exchange 2013 server in two different sites (Production and DR). After adding the Database copy in ECP console it displays Database copy status unknown for the DR exchange server. Issue is strange…
This tutorial will give a an overview on how to deploy remote agents in Backup Exec 2012 to new servers. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as connecting to a remote Back…
This tutorial will walk an individual through locating and launching the BEUtility application and how to execute it on the appropriate database. Log onto the server running the Backup Exec database. In a larger environment, this would generally be …

860 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question