Solved

Software running under VMWare receiving TCP/IP Errors

Posted on 2013-01-16
15
917 Views
Last Modified: 2014-08-05
Wondering if this sounds familer to anyone.

Have a client that is moving to a new hosted server setup at Infinitelyvirtual.com

I don't know what the host hardware or OS is.   Guest OS is Windows 2008 Server R2.

They are currently using Gentran 5.3, which is 64 bit compliant.

Gentran is installed on one VM (a "app" server) and is talking to SQL Server (2008 R2) on another VM.  The other VM runs nothing but SQL Server.

Gentran is reporting a low level error:

ODBC: MFC database exception in Program/RETCODE: Server/-1State:08S01,Native:11,Origin:[Microsoft][ODBC SQL Server Driver][TCP/IP Sockets]
State:01000,Native:10054,Origin:[Microsoft][ODBC SQL Server Driver][TCP/IP Sockets]
 General network error. Check your network documentation.
ConnectionWrite (send()

which basically says there's a network problem.  

 Checking the VMWare forums, others have had this issue as well (not Gentran specifically, but with apps in general talking to a SQL Server).

 We've done quite a bit of troubleshooting on this already, and have gotten no where.  Most of the VMWare forum threads talk about enabling/disabling various TCPIP settings.  

 We've already tried RSS, Chimney Offload, and NET DMA changes on the adapter/OS and nothing's made a difference.

 We do have this same version of Gentran running on a physical server and have no issues with it, so it seems related to the VM environment somehow.

 I'm going to stop here as I'm not sure what other information to provide.

Jim.
0
Comment
  • 7
  • 4
  • 3
  • +1
15 Comments
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
are the VMs using the VMXNET3 network interface, if not replace and use VMXNET3
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
<<are the VMs using the VMXNET3 network interface, if not replace and use VMXNET3 >>

 Yes they are.

Jim.
0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
Comment Utility
in that case, I would get it escalted to VMware Support, although maybe difficult if it's a VM issue, and you do not own the server.
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
Hey and congrats BTW on KILLING the Expert of the year awards.

Don't think I've ever seen one members name up in so many places (and I've been here a while<g>).

Jim.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Jim

Thanks for your kind comments!

Andy
0
 
LVL 8

Expert Comment

by:piyushranusri
Comment Utility
just to undestand my self

did you installed sql native agent on 2008

is vc center or sql server can communicate to others ?
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
It's the SQL ODBC driver that's being used, not the native client.

 Have not seen the errors with other apps yet, but then we do not have anything else running yet outside of Gentran.  We've set a few apps up and checked the configuration, but their not processing anything.  Gentran is the only thing that is actually running constantly, but even it is basically idle at this point.

The errors are very sporadic getting one or two every couple of days.

 We see nothing like that in our current production environment which is based on physical servers running Windows Server 2003 and SQL Server 2000.

 The Gentran database on the VM SQL Server is running in compatibility mode.  It was not upgraded to 2008.

  Were not sure at this point if it is an app problem with Gentran or with the VM setup in general.  With nothing else really running, it's hard to say.

  IBM says Gentran works fine with Windows Server 2008 R2 and SQL Server 2008.  But Ifinitley Virtual did some network tests and could find nothing wrong.  Only abnormaility was in monitoring pings, which would occasionally show times >50ms and very infrequently show times >100ms.  To me, that seems to indicate a network issue.  I've never seen a physical network react like that.  But this is the first time I'm working with VMWare and Windows VM's.

  We also setup a new VM for testing on a different VM network without a firewall and ran it all on a single box and still saw the same errors.  That seems to indicate something with the TCP/IP setup within a single VM.

  As I mentioned above, searches of the VMWare fourms show posts with others having this exact same problem and most "solutions" were trial and error of the TCP/IP settings under Windows or the NIC at both physical and VM levels.  I have played with all those and gotten no where; wish the darn thing would outright break!

 We also did move the VM to a different physical server with no difference.

 Our foucs now is to get other software running and see if errors pop anywhere else, but in t he meantime I thought I'd post in case someone had run into something like this before.  I am going to try the VMWare forums next.

Jim.
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 19

Expert Comment

by:compdigit44
Comment Utility
Hi JDettman,

1) Is AV running on the app server or DB server?
2) Is DEP disabled on the app and DB server?
3) What version of Vmware is the host running, please include the build id?
4) Are both VM's? on the same vSwitch? Also, are you using a Standard or vDS switches.
5) Are you using vLans?
6) Please list the type of physical nics you are using including driver version build numbers.
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
<<1) Is AV running on the app server or DB server?>>

 Not at present.

<<2) Is DEP disabled on the app and DB server?>>

 I tried to disable it, no difference

<<3) What version of Vmware is the host running, please include the build id?>>

   Have no idea.  These are hosted servers and I don't have access to anything outside of the VM.  I can't see the physical servers in any way.

<<4) Are both VM's?>>

  Yes.

<< on the same vSwitch? Also, are you using a Standard or vDS switches.>>

  Don't know, but the hosting company has already tried a test server on a seperate network to address anything like that and it made no difference.

<<5) Are you using vLans?>>

  Not sure.

<<6) Please list the type of physical nics you are using including driver version build numbers. >>

  Don't have access to the physical servers.

Jim.
0
 
LVL 19

Expert Comment

by:compdigit44
Comment Utility
1)Are any other applications having problems accessing the SQL server?
2) I'm guessing the hosting company is not seeing any network related problems on the host server logs correct?
3) Any errors in the Windows event logs?
4) You could install Network Monitor to view the traffic between the client app and sql server.
5) You are able to ping the SQL server from the app server correct?
6) have you tried running pathping or tracert
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
<<1)Are any other applications having problems accessing the SQL server?
>>

  As mentioned, there's not much else running on these servers yet, but no, so far nothing else has popped up.

<<2) I'm guessing the hosting company is not seeing any network related problems on the host server logs correct?
>>

  Correct

<<3) Any errors in the Windows event logs?>>

  No, not outside of the Gentran errors.  

<<4) You could install Network Monitor to view the traffic between the client app and sql server.>>

  And look for what exactly?

<<5) You are able to ping the SQL server from the app server correct?>>

  Yes.  Basic connectivity is not the problem.  These TCPIP errors are very sporadic and random.  99% of the time, everything is fine.

<<6) have you tried running pathping or tracert >>

  We've done some ping logging and the only odd thing is that every so often, we'll see a ping time jump to +50 ms, and then every so often jump to +100 ms for a single ping.  That seems odd to me, since nothing is really running and these machines are all supposed to be on the same network.

Jim.
0
 
LVL 19

Expert Comment

by:compdigit44
Comment Utility
In regards to Network Monitor you would want to look for things like the retransmit of packets or anything in the packet details panel that contains the word error..

In SQL Config Manager, the server is set to allow remote connections correct?
0
 
LVL 57

Author Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
<<In SQL Config Manager, the server is set to allow remote connections correct? >>

 Yes.  Again, it is not basic connectivity that is the issue.  The software works the way it should 99.9% of the time.

Jim.
0
 
LVL 19

Expert Comment

by:compdigit44
Comment Utility
As a test, have to tried to increase the connection time-out settings on the SQL server
0
 
LVL 57

Author Closing Comment

by:Jim Dettman (Microsoft MVP/ EE MVE)
Comment Utility
Never got to the bottom of this...still having the issues and am going to escalate to VMWare support.

Jim.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
When we have a dead host and we lose all connections to the ESXi, and we need to find a way to move all VMs from that dead ESXi host.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now