[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 927
  • Last Modified:

SSLSocket Performance Issues when run inside of JBoss

Two separate connections.  One had the app started by a JBoss web app with a static call to its "main" function.  The other was launched directly in the OS.

When doing a SSLSocket connection to transfer some data, one uses a bunch of other classes, and runs very slowly and high CPU usage.  The other runs fast.

The OS in both cases is XP SP3 running inside VM-Ware.

*FAST*
Thread [demo:(2)-192.168.0.217 (stor_handler)] (Suspended)      
      SocketOutputStream.socketWrite0(FileDescriptor, byte[], int, int) line: not available [native method]      
      SocketOutputStream.socketWrite(byte[], int, int) line: 92      
      SocketOutputStream.write(byte[], int, int) line: 136      
      OutputRecord.writeBuffer(OutputStream, byte[], int, int) line: 295      
      OutputRecord.write(OutputStream) line: 284      
      SSLSocketImpl.writeRecordInternal(OutputRecord) line: 734      
      SSLSocketImpl.writeRecord(OutputRecord) line: 722      
      AppOutputStream.write(byte[], int, int) line: 59      
      VFS_FTP$1$OutputStreamSocketCloser.write(byte[], int, int) line: 917      
      VFS_URL.write(byte[], int, int) line: 603      
      STOR_handler.run() line: 438      
      Thread.run() line: 619      

*SLOW*
Thread [demo:(3)-192.168.0.217 (stor_handler)] (Suspended)      
      System.arraycopy(Object, int, Object, int, int) line: not available [native method]      
      SunJCE_k.a(byte[], int, int, byte[], int) line: not available      
      SunJCE_f.a(byte[], int, int, byte[], int) line: not available      
      DESedeCipher.engineUpdate(byte[], int, int, byte[], int) line: not available      
      Cipher.update(byte[], int, int, byte[], int) line: not available      
      CipherBox.encrypt(byte[], int, int) line: 141      
      OutputRecord.encrypt(CipherBox) line: 197      
      SSLSocketImpl.writeRecordInternal(OutputRecord) line: 733      
      SSLSocketImpl.writeRecord(OutputRecord) line: 722      
      AppOutputStream.write(byte[], int, int) line: 59      
      VFS_FTP$1$OutputStreamSocketCloser.write(byte[], int, int) line: 917      
      VFS_URL.write(byte[], int, int) line: 603      
      STOR_handler.run() line: 438      
      Thread.run() line: 619      



Why did one choose the seemingly alternate method stack?  Is there some other library that is getting loaded and overriding the built in SUN ones?

This is JBoss 4.0.1sp1, and Java 1.6_07.  JBoss is only present on the slow one as mentioned above.

This is killing the transfer as it goes from 500K/sec over the WAN to under 200K/sec.  The CPU usage is pretty dramatic too.

Not all ciphers cause this slowdown in the JBoss one.  For instance, the AES cipher runs equally fast on both systems.

In both cases, the cipher list that is allowed for the SSL Socket has been limited down to the 3DES EBE CBC SHA cipher.  This cipher is required to be used as the other end only supports this cipher.  Other FTP clients run on this same JVM, whether they be Java based, or native clients all get good performance.  Its only the app run from inside of JBoss that uses roughly 50% of the CPU and gets terrible transfer speeds.

Please help!
0
benspink
Asked:
benspink
  • 5
  • 4
1 Solution
 
objectsCommented:
looks like the first is writing directly to stream, and the encryption must be being done elsewhere, whereas the 2nd uses JCE directly
0
 
objectsCommented:
> Why did one choose the seemingly alternate method stack?

though how do you know the two are at the same point in processing
0
 
benspinkAuthor Commented:
Every time I pause the thread, its the same stack trace.  It can't be chance that I always catch it this way.

The slow one (run from JBoss) is always in the middle of doing encryption, while the fast one has apparently already always finished encryption and has moved on to writing the data.

So why was one fast, and one slow when its the exact code being used in both cases?

I did verify via WireShark that the data in the fast one is being encrypted too.
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
objectsCommented:
> Every time I pause the thread, its the same stack trace.  It can't be chance that I always catch it this way.

still doesn't imply they are doing different paths

suggest using a debugger to step thru the fast one

> So why was one fast, and one slow when its the exact code being used in both cases?

not really enough details to be able to say. start with checking the vm startup options
0
 
benspinkAuthor Commented:
I can't debug the internal Sun JVM code.  As you can see in the stack traces, its sun code starting at the 5th line up, and things have different tasks going on starting at about the 7th line from the bottom.

My real issue is, how can the same library, loaded from two different places, which just uses a standard SSLSocket, has drastically different results?

The JVM is started with nothing special.  No overrides for anything SSL, or encryption related, etc.  There is no bouncycastle installed, or other providers.

I printed out the Security.getProviders() list and it was identical in both situations.

I agree there is no guarantee its using different paths, but the fact that one is much much slower, and uses a lot more CPU does indicate it is doing something different.

I can't however tell how, what, or why it would be doing somethign different to the exact same SSLSocket.getOuputStream.write() function call.  (not literally that...)
0
 
objectsCommented:
> I can't debug the internal Sun JVM code.  

why not?  you have a stack trace of it, so you should be able to step into it

> The JVM is started with nothing special.  No overrides for anything SSL, or encryption related, etc.  There is no bouncycastle installed, or other providers.

was referring to command line startup options


profiling may also give some clues as to where it is spending the time
0
 
benspinkAuthor Commented:
I have searched, and I cannot find any source code that matches the line numbers I am seeing for the starting point of where I really need to debug:  SSLSocketImpl.writeRecordInternal

Is there some trick to finding out what version of that class was being used so I at least know what source code revision I am after?  Its Sun's source code...in both instances.  These are built in Classes for JDK 1.6_07.

Yes, I can debug into it, but I am just stepping blindly in eclipse without the source.

I know you meant the command line options, but they really are not interesting at all.  Tomorrow though, I will do another test, and make sure the stand alone app is started with all the same arguments as JBoss was.  There is a slim chance I suppose these could degrade this particular cipher's performance...
0
 
benspinkAuthor Commented:
The answer is the -Xint flag in the default command line args affects how the JVM operates.  -Xint means use interpreted mode only, and never compile the byte code to native code.  As as a result, Java is slow.  The stack traces was directly showing me that one was using native code and happening so fast that I never could catch it, which the other was doing interpreted code every time.

So your question about the command line led me in the right direction.

Can you post some relevant info about the .Xint flag, and the issues that can be expected with JBoss by not using it?  I'll then accept that as the answer.

I appreciate your determination in questioning the command line args as I had just ignored that -Xint flag as it didn't seem important... :)
0
 
objectsCommented:
"Operate in interpreted-only mode. Compilation to native code is disabled, and all bytecodes are executed by the interpreter. The performance benefits offered by the Java HotSpot Client VM's adaptive compiler will not be present in this mode. "

No idea why jboss would be running with that option enabled


0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 5
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now