Link to home
Start Free TrialLog in
Avatar of krakatoa
krakatoaFlag for United Kingdom of Great Britain and Northern Ireland

asked on

Socket connection - what conditions could give rise to the following exception :

What conditions could give rise to the following exception please :

java.net.ConnectException: Connection timed out: connect
        at java.net.DualStackPlainSocketImpl.connect0(Native Method)
Avatar of gsk
gsk

Hi krakatoa,

It seems you are writing server socket programme in java , need to increase response time.please use ping command to check whether
server is alive on cammnd line(cmd) like ping IP address.

thanks
Avatar of krakatoa

ASKER

I know the server is alive - it's another machine on my network and is responsive to my keepalives to it which are running on another socket with it.
please start debugging the code.Is there any firewall there,if so please diable it.please run the program in debug mode,f5,f6 commands hope you know.if any admin guy ask him to disable firewall and then pls try.
I’m not quite sure what role a firewall would be playing when I’m already connected to the other peer on another socket.
already connected to the other peer on another socket.

Are you saying this other socket is connected to the same port number on the server? Or at a different port? Is the server that you are trying to connect to also written in Java or something else?
Connected to another port yes. The server is in Java too.
.Is there any firewall there,if so please diable it.please run the program in debug mode,f5,f6 commands hope you know.if any admin guy ask him to disable firewall and then pls try.

This is nothing to do with firewalls. I am my own admin guy. The issue is one of architecture.
Hi Krakatoa,

can you please share whole programme or java files.
Actually, the original exception no longer applies. These are SSL sockets and the issue is failed handshaking.
so is it resolved.
Well no. Is it your area?
Have you well checked SSL certificate keystore values and configuration.please do that .The server and client should have same authrotiy SSL signed.does it helps
to add further SSL certificates should be properly handled and keystore values should be properly configured thats why server is not trusting client.The server and client should have certificate from same authority
I thought the client only needs to see the server's credentials, and the client doesn't need its own cert?
I thought the client only needs to see the server's credentials, and the client doesn't need its own cert?

It is optional, if you use the below line in the server code (server being an SSLServerSokcet object), then the client needs a cert, but otherwise, no cert required.

server.setNeedClientAuth(true);



These are SSL sockets and the issue is failed handshaking.

Any stack trace? More detailed info? Is the other port (the one that is working) using SSL too?
Thnx mccarl.

The other port is not SSL. That connection is fine.


Yes, I alrady had "setNeedClientAuth(false);" in place. Seems not to have had / still doesn't have, any effect.

As to stack traces and debugging output, there is plenty. Here is the last part of the debug output from the client side, without all the "adding certificates" output, which takes up screenfuls of space. Right after it is the top of the Exception (which now seems to be a timeout instead of a previous Connection Refused, or handshake failure.) This exception has changed most likely because I put the same keystore loading code on the client as well as the server. When I omit it from the client side, the error seems to be the handshake failure.

  Algorithm: RSA; Serial number: 0x33af1e6a711a9a0bb2864b11d09fae5
  Valid from Thu Aug 01 13:00:00 BST 2013 until Fri Jan 15 12:00:00 GMT 2038

keyStore is :
keyStore type is : jks
keyStore provider is :
init keystore
init keymanager of type SunX509
trigger seeding of SecureRandom
done seeding SecureRandom
java.net.ConnectException: Connection timed out: connect
        at java.net.DualStackPlainSocketImpl.connect0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)


I can post some code if it would help.
Yeah, the code would be useful, especially if it is small and self-contained enough that we can run it, both server and client. If not, can you make some small sample code that exhibits the same errors?
Here is the server and client code, cutting out as much other code as I could. Two machines are needed unless one for server - which you'll start by "java SSL_CS S" and enter, and the client with the debug line : " java -Djavax.net.debug=ssl SSL_CS " (copy and paste those to ensure you get the underscores). However, they'll most likely need compiling, right and you are ok with that I'm sure.

The client command prompt should output a final fatal handshake error.) I've no idea why my two System.out.println() statements to get the state of the output and input streams for the SSLSocket, print out just "true" or "false", when I coded for a fuller string explanation with them, but anyway . . .)

import java.net.*;
import java.io.*;
import java.util.*;
import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.nio.file.*;
import java.nio.ByteBuffer;
import java.nio.channels.*;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
import javax.net.*;
import javax.net.ssl.*;
import java.security.KeyStore;
import java.security.Security;
import java.nio.ByteOrder;
import java.security.Provider;


import java.util.stream.Stream;

 
 class SSL_CS extends JFrame implements ActionListener{
 
 JButton showfilechooser;
 JTextArea ipBox;
 JFileChooser jfc;
 String targetIPAddress;
 boolean notconnected = true;
 ServerSocket serversocket;
 ServerSocket servermonitorsocket;
 Socket socket;
 Socket aliveSocket;
 SSLSocket sslSocket;
 SSLSocket sslSocketServ;
 SSLSocket sssock;
 Client client;
 ConnectionMonitor connectionmonitor;
 boolean aclientrunning;
 String myIP;
 File[] outgoingFiles;
 File[] incomingFiles;
 ObjectOutputStream oos;
 ObjectInputStream ois;
 JButton joinButton;
 Thread thread;
 JRadioButton jrb;
 boolean loopbacktry;
 PrintWriter pW;
 BufferedReader bR;
 Object syncObj;
  
   
 final int BASECHUNK = 32768;
  
 static boolean engaged = false;
 
    
    public static void main(String[] args){
                
            SSL_CS ante = new SSL_CS(args);
            
    }
 
 
    public SSL_CS(String[] args){
    
            
            
            init();
            
            
                new Thread(new ServDriver()).start();
            
            
    
    }
 
    public void actionPerformed(ActionEvent ae){
 
        String buttonName = ((JButton)ae.getSource()).getName();
        
        switch (buttonName) {
        
            case "helpbutton" :  
                
                JOptionPane jop = new JOptionPane();
                
                
                jop.showMessageDialog(this, 
                " *  To connect to a partner, enter their IP address into the white box in the bottom, middle of the main window, and push Enter.\n * Then click \"Connect Out\"\n *  If they are connecting to YOU first, let them know the IP address given above.","The IP Address your partner needs is "+myIP,JOptionPane.INFORMATION_MESSAGE);
                
                break;
                                 
            case "filechooser" :
                
                if(engaged == true){break;}
                
                
                if(( sslSocket== null)||pW.checkError()){return;} // use in production
   
                          
                jfc = new JFileChooser(new JFileChooser().getFileSystemView());
                jfc.setMultiSelectionEnabled(true);
      
                jfc.setSize(new Dimension(500,400));            
                jfc.setVisible(true);
                jfc.setCurrentDirectory(new File("C:/TransferHolder"));
                
                int yes = jfc.showOpenDialog(null);
                
                if (yes == JFileChooser.APPROVE_OPTION) {
                
                    outgoingFiles = jfc.getSelectedFiles();
                                                         
                
                    
                    
                                    
                }
               
                break;
                
            
            case "joinbutton" :
            
            if(ipBox.getText().equals("")){return;}
            if(loopbacktry){ipBox.setText("");loopbacktry=false;return;}
                   
            GetSocket gs = new GetSocket();
                  
                                    
                    
                    gs.getSock(targetIPAddress,40000);
                                
                    if(aclientrunning ){
                    
                            thread = null;client.keeprunning = false; client = null;
                            client  = new Client();
                            thread = new Thread(client);
                            thread.start();       
                    }
                   
                    else {
                            
                            client = new Client();
                            thread = new Thread(client);
                            thread.start();
                            
                    } 
                            
                    aclientrunning = true;
                          
                                                
                if(gs.getLSocket(targetIPAddress,40001)){
                    aliveSocket = gs.lSock;
                    ConnectionMonitor cm = new ConnectionMonitor();
                    Thread cmThread = new Thread(cm);
                    cmThread.start();
                }
           
                gs = null;
                
                break;
        }       
       
    }
    
 
    void init(){
    
        this.setSize(new Dimension(550,185));
        this.setLocationRelativeTo(null);
        this.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        this.setResizable(false);
        
       // showfilechooser = new JButton("Show Filechooser");
       // showfilechooser.setName("filechooser");
       // showfilechooser.addActionListener(this);
        
        this.add(new JPanel(), BorderLayout.NORTH);
        this.add(new JPanel(), BorderLayout.WEST);
        this.add(new JPanel(), BorderLayout.EAST);
        
       
        JPanel jp = new JPanel();
        this.add(jp, BorderLayout.CENTER);
        //jp.add(showfilechooser);
        JButton helpButton = new JButton("Help");
        helpButton.setName("helpbutton");
        jp.add(helpButton);
        helpButton.addActionListener(this);
         
                
        ipBox = new JTextArea();
        ipBox.setMaximumSize(new Dimension(90,18));
        ipBox.addKeyListener(new IPInputListener());
        ipBox.setLineWrap(true);
        ipBox.setVisible(true); 
        
                
        JPanel southPanel = new JPanel();
        southPanel.setLayout(new GridLayout());
        southPanel.setPreferredSize(new Dimension(this.getWidth(),19));
        
        this.add(southPanel,BorderLayout.SOUTH);
        
        jrb = new JRadioButton("Idle",false);
        jrb.setEnabled(false);
        jrb.setOpaque(true);
        jrb.setBackground(java.awt.Color.GRAY);
        jrb.setForeground(java.awt.Color.BLACK);

        joinButton = new JButton("Connect Out");
        joinButton.setName("joinbutton"); 
        joinButton.addActionListener(this);
                
        southPanel.add(jrb);
        southPanel.add(ipBox); 
        southPanel.add(joinButton);  
  
        this.setVisible(true);
    
    }

        
    public class GetSocket{
    
                
        public SSLSocket sssock;
        
        public Socket lSock;
        
        public SSLSocket getSock(String host, int port){
                   
                                        
                    try{
                        if(host.equals(InetAddress.getLocalHost().toString().substring(InetAddress.getLocalHost().toString().indexOf("/")+1))/*||(host.equals(myIP))*/){return null;}
                    }catch(Exception ukhe){}
                    
        
                    try{                      
                                             
                        SocketFactory ssl_s_f = (SSLSocketFactory)SSLSocketFactory.getDefault(); //extends SocketFactory.
                        
                        try{
                       
                        sslSocket = (SSLSocket) ssl_s_f.createSocket(InetAddress.getByName(host),port);
                        
                        
                        }catch(IOException ioexxx){ioexxx.printStackTrace();}
                                                                                                   
                    }catch(Exception unhe){unhe.printStackTrace();ipBox.setBackground(java.awt.Color.RED);ipBox.setText("No connection to that IP");return null;}
                    
                    return sslSocket;
        
        }
        
        public boolean getLSocket(String host, int port){
        
            try{
                
                        lSock = new Socket();
                        lSock.connect(new InetSocketAddress(host, port), 20000);
                
            }catch(Exception unhe){ipBox.setBackground(java.awt.Color.RED);ipBox.setText("No connection to that IP");return false;}
            
            return true;
        }
    
    }
  
    
    private class ConnectionMonitor implements Runnable{
    
        boolean talking;
    
        private void clearUp(){
               
                        jrb.setSelected(false);
                        jrb.setBackground(java.awt.Color.GRAY);
                        jrb.setOpaque(true);
                        jrb.setText("Idle");
                        ipBox.setText("");
                        client.keeprunning = false;
                        Thread.yield();
                         
                        try{
                        
                        pW.close();
                        bR.close();
                        
                        client = null;
                                              
                        }catch(Exception bye){/*bye.printStackTrace();*/}
        
        }
    
        public void run(){
        
                talking = true;
        
                try{
                    pW = new PrintWriter(new OutputStreamWriter(aliveSocket.getOutputStream(), "UTF-8"), true);
                    bR = new BufferedReader(new InputStreamReader(aliveSocket.getInputStream(), "UTF-8"));
                    
                    
                try {
                
                while (talking) { //*****maintain connection*****
           
                  
                        pW.println("noOp");
                                      
                        try{Thread.sleep(10000);}catch(Exception woken){}
                    
                        if (pW.checkError()) {
                        
                            clearUp(); 
                        }
                    
                        //System.out.println("Monitor polling . . . "); //DEBUG
                 
                                 
                }
                
            } catch (Exception intsoc) {}
            }catch(Exception setupex){}
            
        }
    
    }
    
    public class Client implements Runnable{
    
        private boolean keeprunning;
                  
        
        Client(){ 
                 
                jrb.setSelected(true);
                jrb.setOpaque(true);
                jrb.setBackground(java.awt.Color.GREEN);
                jrb.setText("Live");
                
                                
                ipBox.setBackground(java.awt.Color.WHITE);
                try{ipBox.setText("Partner :"+sslSocket.getRemoteSocketAddress());}catch(Exception tex){tex.printStackTrace();}
                       
        }
            
    
        public void run(){
        
                                    
                    
                try{      
                    oos = new ObjectOutputStream(sslSocket.getOutputStream());
                    ois = new ObjectInputStream(sslSocket.getInputStream()); 
                    
                    
                }catch(IOException ioex){System.out.println("It is "+oos==null+" that oos is null");System.out.println("It is "+ois==null+" that ois is null");}                      
               
               
               
                              
            
                                 
            }//run
          
        }//client   
    
     
    
    class ServDriver implements Runnable{
 
        public void run(){
 
            try{
            
                            
          KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());   

          keyStore.load(keystoreInput, "lssetna".toCharArray());

          // initialize a key manager factory with the key store
          KeyManagerFactory keyFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());    
          keyFactory.init(keyStore, "lssetna".toCharArray());
  
          // get the key managers from the factory
          KeyManager[] keyManagers = keyFactory.getKeyManagers();
        
          // Now get trustStore
         
          KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());   
          trustStore.load(truststoreInput, "lssetna".toCharArray());
          
           
            ServerSocketFactory ssl_ss_f = SSLServerSocketFactory.getDefault();
            ServerSocket sslServerSocket = (SSLServerSocket) ssl_ss_f.createServerSocket(40000);
            
            //System.out.println("ServerSocket up . . . ");
          
                        
            servermonitorsocket = new ServerSocket(40001);
                      
            while(true){   
        
                
                sslSocket = (SSLSocket)sslServerSocket.accept();
                //System.out.println("Connection established");
                sslSocket.setNeedClientAuth(false); 
                                             
                client = new Client();
          
                new Thread(client).start();
                
                aliveSocket = servermonitorsocket.accept(); 
              
                connectionmonitor = new ConnectionMonitor();
                
                Thread thr = new Thread(connectionmonitor);
                thr.setDaemon(true);
                thr.start();
                        
            }
        
            }catch(Exception sockex){}
    
        }
 
    }
          
 
        class IPInputListener extends KeyAdapter{ // this is not a cast-iron IP parser, just a format entry helper for users.
     
            public void keyReleased(KeyEvent k){
                
                if((ipBox.getBackground()==Color.GREEN)||(ipBox.getBackground()==Color.RED)){ipBox.setBackground(Color.WHITE);}
                if(!((k.getKeyChar()=='.')||(Character.isDigit(k.getKeyChar())))&&!(k.getKeyCode()==KeyEvent.VK_ENTER)){ipBox.setText("");ipBox.setBackground(Color.RED);try{Thread.sleep(300);}catch(Exception waiting){}return;}
                else if(ipBox.getText().length()>15&&!(k.getKeyCode()==KeyEvent.VK_ENTER)){ipBox.setText("");ipBox.setBackground(Color.RED);try{Thread.sleep(300);}catch(Exception waiting){}return;}
                
                if(k.getKeyCode()==KeyEvent.VK_ENTER){
                               
                    targetIPAddress = ipBox.getText();
                    
                    if(Pattern.matches("(?<![0-9a-zA-Z.])(?:\\d{1,3}\\.){3}\\d{1,3}(?![0-9a-zA-Z.])", targetIPAddress.trim())){
                    
                        ipBox.setText(ipBox.getText().trim()); //trim MUST be present.
                        ipBox.setBackground(Color.GREEN);
                        targetIPAddress = ipBox.getText();
                                                
                    }
                    else{ipBox.setText("");ipBox.setBackground(Color.RED);try{Thread.sleep(500);}catch(Exception waiting){}}
                }
                if(ipBox.getText().indexOf("127.")==0){
                    ipBox.setText("Loopback not addressable.");loopbacktry=true;
                }
                 
            }
        
        }
 }

Open in new window


The keystore and cert need to be borne in mind if compiling.
ASKER CERTIFIED SOLUTION
Avatar of mccarl
mccarl
Flag of Australia image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ok, wow. I'll take a look at all that and be back.
Right, well the result here looks a lot more promising, although I'm certainly not there yet and there's something I've done wrong or is still missing.

Your -Djavax sample debug output above is mirrored here from line 2 downwards, but line 1 is absent.

The issue now is a handshake alert stating       "certificate_unknown"      instead of the earlier, anonymous-looking outright handshake failure. So is there something still not kicking in regarding the cert, do you think? Should I redo the cert., or post the new code I now have ? Thanks mccarl.

Debug:

%% Invalidated:  [Session-13, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256]
Thread-27, SEND TLSv1.2 ALERT:  fatal, description = certificate_unknown
Thread-27, WRITE: TLSv1.2 Alert, length = 2
Thread-27, called closeSocket()
Thread-27, handling exception: javax.net.ssl.SSLHandshakeException: sun.security
.validator.ValidatorException: PKIX path building failed: sun.security.provider.
certpath.SunCertPathBuilderException: unable to find valid certification path to
 requested target

Open in new window


By the way, my certificate is self-signed if I didn't make that so clear already.
Yeah, I think now that it is just something to do with your keys/certs. Can you describe (or better yet, copy paste) the steps you took to generate? And also, posting the new code might help.

To give you an idea, what I did was just to generate a new key pair using keytool into a fresh new .jks file...
keytool -genkeypair -keyalg RSA -keysize 2048 -keystore test.jks -storepass lssetna

Open in new window

You can put some details in but they aren't technically necessary for getting the connection to work.

And then I load the "test.jks" file into a KeyStore to get the server SSLContext's KeyManagers. I also use the same "test.jks" file to load into a KeyStore to initialise the client SSLContext's TrustManagers.

I did notice in the code above that you have both a .jks file and a .crt file, so my guess is that maybe something is going on around these files, so that's why if you describe how you got those two files, it may help. Also, maybe have a go at what I said just above to see if you can get it working, just as a test.

Is the code that you are writing meant just for your usage? Or is it something that will be distributed to others, etc? This will have a bearing on how you eventually manage your keys/certs. At the moment, for testing, you can just use above procedures but this could be something to give some thought to as it may require code changes depending on how you want it all set up. If you are unsure, you can describe how this code will eventually be used and I can give some suggestions on how you might want to structure everything.
Made a new keystore with the params you outlined and the same exception occurs of certificate_unknown at handshake time.

I also stopped using the cert (or putting it in the .jar file at least).

I am not entirely sure of the actual steps I took to make the original keystore and cert., but wouldn't they now be replaced by the test.jks anyway? I do remember suppying an alias though, and exporting the cert for some reason, but again, I've omitted the cert this time from the .jar and there is still no connection.

I intend to use the app with others, yes, eventually. But if it can be got working on the lines we are trying now, that would be enough for the time being, just to know how it should be configured. So I'll perhaps scrap the cert and the keystore and try once more ?

Here is the code I've used on your guidelines :

Effectively, this is the client :

public class GetSocket{
    
                
        public SSLSocket sssock;
        
        public Socket lSock;
        
        public SSLSocket getSock(String host, int port){
                   
                                        
                    try{
                        if(host.equals(InetAddress.getLocalHost().toString().substring(InetAddress.getLocalHost().toString().indexOf("/")+1))/*||(host.equals(myIP))*/){return null;}
                    }catch(Exception ukhe){}
                    
        
                    try{   
                        
                        KeyStore keyyStore = KeyStore.getInstance(KeyStore.getDefaultType());   

                        keyyStore.load(keyInput, "lssetna".toCharArray());

                        // initialize a key manager factory with the key store
                        KeyManagerFactory keyyFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());    
                        keyyFactory.init(keyyStore, "lssetna".toCharArray());
                                              
                        
                        TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
                        trustManagerFactory.init(keyyStore);
                        TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
                        
                        SSLContext ctxx = SSLContext.getInstance("SSL");
                        ctxx.init(null, trustManagers, null);                                             
                        SSLSocketFactory ssl_s_f = ctxx.getSocketFactory();
                        
                        
                                    /////////////////////
                    
                                             
                        SocketFactory s_f = (SSLSocketFactory)SSLSocketFactory.getDefault(); //extends SocketFactory.
                        
                        try{
                       
                        sslSocket = (SSLSocket) s_f.createSocket(InetAddress.getByName(host),port);
                        
                        
                        }catch(IOException ioexxx){ioexxx.printStackTrace();}
                                                                                                   
                    }catch(Exception unhe){unhe.printStackTrace();ipBox.setBackground(java.awt.Color.RED);ipBox.setText("No connection to that IP");return null;}
                    
                    return sslSocket;
        
        }
        
        public boolean getLSocket(String host, int port){
        
            try{
                
                        lSock = new Socket();
                        lSock.connect(new InetSocketAddress(host, port), 20000);
                
            }catch(Exception unhe){ipBox.setBackground(java.awt.Color.RED);ipBox.setText("No connection to that IP");return false;}
            
            return true;
        }
    
    }

Open in new window


and this is the server side :

class ServDriver implements Runnable{
 
        public void run(){
 
            try{
                            
          KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());   

          keyStore.load(keystoreInput, "lssetna".toCharArray());

          // initialize a key manager factory with the key store
          KeyManagerFactory keyFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());    
          keyFactory.init(keyStore, "lssetna".toCharArray());
  
          // get the key managers from the factory
          KeyManager[] keyManagers = keyFactory.getKeyManagers();
          
          SSLContext ctx = SSLContext.getInstance("SSL");
          ctx.init(keyManagers, null, null);
          SSLServerSocketFactory ssl_ss_f = ctx.getServerSocketFactory();

          ServerSocket sslServerSocket = (SSLServerSocket) ssl_ss_f.createServerSocket(40000);
            
          //System.out.println("ServerSocket up . . . ");
          
                        
            servermonitorsocket = new ServerSocket(40001);
                      
            while(true){   
        
                
                sslSocket = (SSLSocket)sslServerSocket.accept();
                //System.out.println("Connection established");
                sslSocket.setNeedClientAuth(false); 
                                             
                client = new Client();
          
                new Thread(client).start();
                
                aliveSocket = servermonitorsocket.accept(); 
              
                connectionmonitor = new ConnectionMonitor();
                
                Thread thr = new Thread(connectionmonitor);
                thr.setDaemon(true);
                thr.start();
                        
            }
        
            }catch(Exception sockex){}
    
        }
 
    }

Open in new window


(I left out the rest as it doesn't relate to SSL as you know from the earlier post of the whole code).
Ok, I can see your issue. In your client code you are creating TWO SSLSocketFactory objects, the one on line 35 is correct as it comes from the initialised SSL context, but then on line 41 you create another one, but it's just the default and so doesn't contain any of your trust manager setup.

So get rid of line 41, and change line 45 to create the socket from ssl_s_f instead.
Well, that seems to have done the trick alright! I can't believe I duped the SSF, but, then again, my "excuse" is in the Java in a Nutshell description of SSLContext, in which the author states : "... most applications do not need to use thise class directly . . . " and " Most applications use the default SSLSocketFactory and SSLServerSocketFactory objects returned by the static getDefault() methods of those classes".

That made me think that an SSLContext was going to be something for especially esoteric network / socket configuration, and that a 'regular' SSF and SF would be fine for my purposes.

Despite the fact that your comments have solved the issues, I'm still really in the dark as to the role of the certificate in all this, as well as other particles like the alias which was something that always showed up in reading I'd done on using the keytool etc. I wonder too how the user would know that SSL is actually in use, since there is no "https" or padlock icon equivalent shown automatically anywhere afaics.

And this line  : TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();

. . . I can only assume that the trustManager array is populated  - behind the scenes - by the call to init() on the trustManagerFactory, otherwise the array would be empty I would think. The protocols of SSL assignment seem obscure and shadowy compared to the usual declarations assignments and object passing that I would have expected. Is any of that comment relevant? Just FMI, you understand. ;)

There are other questions, but the main hurdle has been dealt with now I hope.
Thanks both.
Glad that it is working now. To the points you've made in the last post...

Most applications use the default SSLSocketFactory and SSLServerSocketFactory objects returned by the static getDefault() methods of those classes

Ok, so I was just following your lead where you were loading the KeyStore in via code. When you are doing that, you need to set the results in a new SSLContext and use that to get your SocketFactory objects. You can use the "default" ones as obtained from the getDefault() methods, but you need to use Java system properties to define their location and passwords, etc. I'm also not sure that if you are doing it this way, if you can specify key/trust stores that are inside your JAR. Anyway, if you want to play with this method, even just for testing or for your own learning, you would be looking to specify these options (note that I haven't tested this with your code though)

-Djavax.net.ssl.keyStore=path/to/keyStore.jks
-Djavax.net.ssl.keyStorePassword=lssetna
-Djavax.net.ssl.trustStore=path/to/trustStore.jks
-Djavax.net.ssl.trustStorePassword=lssetna

Open in new window



like the alias which was something that always showed up in reading I'd done on using the keytool
They are pretty much just names that make referring to keys/certs in the keyStore easier to do. Because your keyStore only contains the one, it doesn't really matter too much.


I wonder too how the user would know that SSL is actually in use, since there is no "https" or padlock icon equivalent shown automatically anywhere afaics.
So that would be entirely up to you. The padlock that you talk about is just a way that browsers starting indicating this. However, the way in which your app works, the connection is always via SSL so you could either just put a permanent indication somewhere in your app, or you would state it in your app's documentation that all communication is via secure sockets. Or something like that...


I can only assume that the trustManager array is populated  - behind the scenes - by the call to init() on the trustManagerFactory
Exactly, and this is the same for the KeyManager factory/array. And yeah, I do agree a bit that some of this may seem to be hidden, or black magic, but it just one of those things that you pick up and then you just know what to do to get it all setup right.


I'm still really in the dark as to the role of the certificate in all this
It would probably be good if I gave you a brief/simplified view of what is going on with the SSL sockets in regards to keys/certs etc. so that you could understand the security implications that various design decisions have on the result. As you may imagine, like a lot of things, there is a compromise between the level of security achieved and other factors, such as easy of use, application architecture, etc. But you really do probably need the basics to then be able to get the higher level stuff. Anyway, let me know when you are happy with the points above, and I will try to come up with a SSL 101 type of thing.
Great work, and thanks again. I'll get this all fixed in my mind and get back when I have. I'll meantime be trying to see if the .jar is really amenable to being accessed effectively for one thing. It looks that way at the moment, although as you have doubts, it's a bit concerning. ;)
Sorry, I was away for the weekend, haven't had a chance to get back to this.

Just to be clear, the doubts that I was having were just around specifying the keystore in a jar via a system property. When you are reading the keystore in with code then there shouldn't be any problems at all.
<<via a system property>>

Ah, right.

Well as you say, there doesn't seem to be any problem getting at those resources in a .jar file. (I added a padlock .jpg in there, and that too can be got at and used in the title bar, for example).
All good then. Do you think me going through a simplified SSL explanation now would be useful, or are you still trying to get everything straight in your mind first?
I'm absolutely up for your explanation, that sure. I would just ask for a couple of days grace still, as I've got a few non-Java, non IT issues on my plate, which when cleared will allow me to regain my focus. :)
mccarl:
I'd be very happy to hear your pointers on SSL as you mentioned. Thanks, k.
Hi mccarl. If you are around, I'd be very glad to get your simplified SSL explanation. ;)
Hey sorry, somehow I missed the notification for the message before. I will get on to that tomorrow (I don't think I could type it all out now on my little phone keyboard)
Thanks very much mc. Understood. ;)
Ok, lets go... Just a note to others reading this, there are very likely some errors (or omissions) in the following, but it is meant to just give an idea of what happens so that the role of keys/certificates can be described and understand the implications in the context of code like the above. Feel free to correct anything here, but the idea is not to go into too much detail.

Firstly, a bit of background on public/private keys, otherwise known as Asymmetric Encryption. Public and Private keys always come as a pair, the two keys are linked in a way that allows the following encryption to take place BUT also are such that you can't (ideally) derive the private key from the public key.

Once you have this key pair, you can perform encryption/decryption of raw data in two directions. You can encrypt with the private key and then decrypt with the public key. And you can also encrypt with the public key and then decrypt with the private key ONLY. Notice from this that you CAN'T encrypt/decrypt again with the same key. This is crucial to everything else described below.

Now when we talk about keys/certificates, the key is specifically the Private key and the certificate is a bunch of data that includes the Public key. The extra data in the certificate is used to establish trust in the public key, I will (hopefully) get to that a bit later. And when we talk about KeyStore we are storing both the Private key and the certificate (and Public key) and with TrustStore we are storing just the certificate (and Public key).

Also, just to cover things off, there is also Symmetric Encryption, where there is only one single key used on both sides to encrypt/decrypt data, therefore requiring both sides to know the key. Why do we use both? Well the asymmetric nature of private/public keys is very useful but it can't be used to encrypt/decrypt large amounts of data, being relatively time-consuming to do this. Whereas, symmetric encryption is relatively fast and can operate on huge streams of data quite efficiently, but requiring both side to know the key raises issues to solve, so that leads us nicely on to...

Ok, so now a brief description of the logical steps taken to establish a secure connection...

1. Client makes a TCP connection to the server, usually on a different port to a normal unsecured connection
2. Server is configured with a KeyStore and so it sends to the client the certificate that is stored within
3. The Client takes steps to establish if it trusts the certificate that the Server sent. There are a number of ways, but for the moment, let's just say that if the Client receives an identical certificate to one stored in its TrustStore, then it does trust the certificate
4. The Client generates a random set of data which will be used later as a key for the Symmetric encryption
5. The Client takes this random set of data (symmetric key) and encrypts it using the public key that was given in the certificate sent by the Server. It then sends this encrypted data to the Server
6. The Server decrypts the data using it's Private key (stored in it's KeyStore) and therefore now has the raw Symmetric key that was generated by the client
7. The Server and Client can now use this shared key and symmetric encryption to freely transmit data

So now a bit more description about the security aspects of those steps, easiest done in reverse. In step 7, the data is transmitted encrypted using the symmetric encryption key. So the security of this key is critical to the security of the communications. This key only comes to existence in step 4 when it was generated on the client, at this stage no-one else knows about it but that's not very useful, so we need to get it to the server. So we encrypt it with the server public key (step 5) and send it. Now anyone can intercept this particular transmission but as detailed in the public/private key description above, ONLY the server private key can decrypt it. But now it has the key too, and the Server and Client can communicate knowing that no-one else can see the raw data going between.

From this, you should be able that ultimately, the security of the Private key, is the most important aspect of all of this!!


Now, I will talk about a few of the general security attacks that may happen that you might want to consider, and that the above attempts to guard against.

1. Passive eavesdropping - passive (or inadvertent) is not really an attack but this is more just stopping from sensitive traffic appearing perhaps in logs, or if someone happened to be viewing traffic and just saw the raw data
2. Active eavesdropping - is more of an attack because it involves someone intentionally logging traffic with the intent to gain access to the raw data going from one party to another
3. Man in the Middle - is where someone intentionally inserts themselves (somehow, there are a few ways) in between the two parties communicating with the intent of either viewing the traffic and/or modifying the traffic
4. Impersonation - where someone connects to another party pretending to be someone else

So for point 1 above, mitigating this is reasonably easy. You can do something as simple as obfuscate the data, i.e. make a change to the data that is easily reversible but on casual observation makes the data look unintelligible, such as just invert the bits in each byte going through the comms channel. You could also encrypt the data (symmetric) but using either a well known key or generate the key at the start and just send it in plain text before starting to encrypt.

For point 2, you should be able to see that the measures taken above would be easily circumvented by someone who is intentionally trying to get to the raw data. So to try and stop this intentional attack, you need to go further. And that is where the SSL type procedures come in. As stated before, the symmetric key is shared in a way where no one else (as long as the private key is kept... private) can get a hold of it, therefore securing the connection.

For point 3 and 4, you need to delve a little deeper. The problem with the way I described the secure connection steps above, is that it isn't scalable in a general sense. Look again at step 3 and you can see that the Client had to specifically import the server certificate into it's TrustStore in order to trust it. Why do we need to trust it? Well, if an attacker can somehow insert itself into the middle of the communications between Server and Client, then it could use it's own private/public key (no need to know the Server private key) and send it's certificate to the Client. If the Client has no way to establish trust, it blindly continues on the steps outlined above, just assuming that it is talking to the real Server, once the secure connection between Client and Attacker is established, the Attacker can make it's secure connection to the Server (just like any Client can) and then it just forwards traffic back and forth, whilst also logging it, or modifying it on the way.

So to mitigate these problems, we need a way to be able to establish "trust" in the certificate sent to the Client but in a scalable way, without needing to preload the certificate into a TrustStore. This is the role of "Certificate Authorities" (CA's), if you here terms like Root CA and Intermediate CA, that's what this is all about. I won't go into things here as it would probably double the length of this post, but if you are interested, let me know and I can go through it.


So what does all this mean, in your context? Well, firstly you have to decide what sort of threats you are trying defend against. I think I may have put this question to you once before (in another question) but hopefully now you see why it is important to make this decision... and that is because there are trade offs to be had between how secure the system is and how complex the system is. You may have a different answer, but my guess is your "sweet spot" might be to defend against points 1 and 2 and leave the risk there of point 3. Also because it appears that your architecture is just a peer-to-peer communication, point 4 doesn't really come in to it. Also, note that I believe (not that I am a major hacker or anything) that point 3 would be fairly complicated/specialised to pull off and therefore a lot less likely to happen.

The code that you were able to get working above, at the moment, is probably only good for defending against point 1. Because the private/public key pair would need to be embedded in the code (or at least the jar) that gets distributed publicly, the security of the private key is lost and so anyone that gets a hold of your software package has the key and therefore can decrypt any communications between any other parties.

So if you aim to cover off point 2 (and not worrying about points 3 & 4) this is how I would do it. At install time (or the first time the software is run, or every time that the software starts), the software generates it's own private/public key pair and then generates a self-signed certificate using that public key. This info can be then loaded into a KeyStore which the existing Server SSL code uses to establish the connection. The only issue with this is that the Client making the connection has no way of establishing trust of this newly generated certificate, so you have a number options.. either you make the Client always trust any certificates received (this is where the risk of point 3 occurring comes in), or you ask the 2 users attempting to connect to establish trust in some other way, or you provide centralised infrastructure to assist in establishing that trust. As I said before, my guess is that the last option there is too complicated for what you need, and maybe the second last option too, and so I would just go for implicitly trusting all certificates sent to the client.

If you are interested in going down some of these paths, at any point, let me know and I can help in terms of coding it up. It's not 100% straightforward, but can be done.

Well I think that is all I can think of at the moment, have a read (and maybe a re-read) and process it all, and if you have questions, let me know!
That's very helpful mc - thank you. I need to read it with the attention it deserves, and I'll be back very soon.
Whew . . . I've been down with a bug . . .  personal one, not a computer one this time. ;)

Can I ask first if you could please clarify in a few words how the certificate figures in the mechanics of client-server authentication . . .  by which I mean . . . I understand that certain credentials are held in the certificate and that these need to be properly present and imparted to the other side, but *where and *when does this happen? See, I've read that the certificate is send to the other party, but in the code we are talking about here at least, that never happens - or does it? Am I wrongly assuming that the "certificate" is sent to the other party as a "file" or stream at least of some kind, when in fact that is not the case? I can't seem to get past this conceptual point before I absorb anything else?

;)
I've read that the certificate is send to the other party, but in the code we are talking about here at least, that never happens - or does it?

It does but it all happens behind the scenes.


but *where and *when does this happen?

So I should have clarified this in the above description, but all the 7 steps that I went through all occur in Java's SSL handling code (not in your code). The 7 steps above starts when the client calls the following code (from your last code posted above)...
sslSocket = (SSLSocket) s_f.createSocket(InetAddress.getByName(host),port);

Open in new window


As for the server side, the server would have blocked on the following code...
sslSocket = (SSLSocket)sslServerSocket.accept();

Open in new window

... and it is within this method that the server side of the negotiation would happen, i.e. it is within this method call where the server would have sent the certificate to the client. So hopefully that answers your where and when. Maybe your next question is how does it know *what to send as the certificate? Well, it sends the certificate that is stored within the KeyStore that we passed to the SSLContext (via the KeyManager array).

Does that help? If you have any further questions about the certificate though, just ask!
Does that help?
Yes, considerably.

-------------

Back to your earlier post with the fuller SSL description -  I am not clear really as to why point 2 is not already covered off by the SSL that has already been established? And also, I take it that by "raw data" you mean the data as transmitted in encrypted form ? If so, surely capturing it per se when it's encrypted is not all that revealing is it ? Surely by then, the encryption has already "done its bit" and the data is as safe as it ever was or ever would be ? After all, breaching the security would need to take place at some point if the SSL were in any sense effective at *any stage? Or am I confusing chickens with geese on this one ?

So I am saying that at whatever stage data might be intercepted or captured, it wouldn't make any difference to how impregnable that data was if it were sheathed in SSL?

I do see the point about open access if the credentials as shared between all recipients of the software, and so authentication could be strengthened by another method of mutual authenticity being met. Maybe that could be a IM interface (encrypted or not) whereby the two parties satisfy themselves that they are who they say they are by sharing some other information. ?
As to the generation of the server certificates in real time, that would be a certain area of interest I'd like to ask you to help me look at. If you think you'd want outline what I should do to get started on it, then please go ahead when you can.  ; )
I am not clear really as to why point 2 is not already covered off by the SSL that has already been established?
Ok, so those 4 points regarding types of attacks were intended to refer to any type of communications, and to show you that SSL can be used to cover off some/all of those types of attacks. So yes, SSL does cover off point 2, and that was exactly the point I was trying to make! ;)


I take it that by "raw data" you mean the data as transmitted in encrypted form ?
No, actually I was meaning the opposite. I meant the raw unencrypted data that you are trying to send to the other party.

Not sure about the couple of questions just after this? Or maybe they were all dependant on you thinking I was talking about the encrypted data, and so they don't apply now. Anyway, if you do still have questions around this now, can you ask them again?


As for runtime generation of server keys/certificates, I have already had a bit of a look at it and as far as I can see there are two main paths to take. 1) is to use classes built-in to (some) JVM's, these are the ones already used by the "keytool" program to generate these keys and such, or 2) is to use a third party library (Bouncy Castle) to do the same. They both should generally work fine and achieve the same outcome, the difference is that 1) will only be available on some JVM's and if the eventual idea is to be able to release this to a wide public audience, then you would have issues in terms of your users being required to use certain JVM's or some users (eg. linux) not having access at all to a JVM that will have the required classes. However with 2) the issue might be that you are trying not to rely on any third party libraries, or if you are happy to use them, this particular one may have licensing terms that are incompatible with how you were hoping to release your software.

All things being equal, if you are not particularly concerned with using the libraries or licensing issues, then my recommendation would be to go with the Bouncy Castle alternative, but let me know what you think and I will look at getting some code working with whichever alternative that you think would best suit.
So yes, SSL does cover off point 2, and that was exactly the point I was trying to make! ;)

It's just that in your "points" comment a few posts ago, you said this :

The code that you were able to get working above, at the moment, is probably only good for defending against point 1.

... so I am now assuming that SSL itself covers Point 2, but the implementation of it in my code does not. I'm a bit confused on that. ; )

The main general query I have with the wider application of SSL is that (at least until my understanding fully solidifies) that at the moment I want to understand whether the data *whilst in transit* is encrypted. I'm (at this very point) not so absorbed with the other important effects and vulnerabilities that SSL can address, but I can't really say I feel assured by what I know now that the data in transit is encrypted. Could you please give me your idea on that particular issue mc ?

As to the realtime certificate issue - I'm not knowledgeable in the slightest about Bouncy Castle, and so I would be a) happy to learn about it, and b) take your guidance on the issue and its alternative, given what you've said about it. But one "strategic" point from me that *may* show the general direction of my thinking is this : whilst the cross-communication between absolutely configured clients and servers would be an issue (that you already rightly pointed out), if there were a way to authenticate both parties before the handshaking, that would be fine with me. It would get around the requirement for a Bouncy C or other approach, and it would satisfy my eventual use-case and application vector - which may not be perfect, but to me seems sufficient. So to sum up this thinking, I'm proposing deployment under the twin conditions being met : 1) SSL traffic being * encrypted on the wire*, even if it is vulnerable to intermediate theft / eavesdropping / etc en route or at source or destination, and 2) that prior to *any* working socket connections, authentication of parties is met by another means, which, at the moment, I am putting my money on reading the HKEY registry presence of a USB memory stick device's ID ^^. In this scenario, before any meaningful SSL connection is attempted, the client and server would exchange this ID, which would be the same on both sides - however the ID must be - or must at least HAVE BEEN - present on the server side to allow full connections to be established. To be clear - I'm not suggesting this form any part of the SSL protocol negotiations.
; ) , k.

^^ That is why I opened this question on EE, to examine whether capturing the ID is feasible :

link
Sorry again for the delay, work has been crazy! ;-)

Ok, I can see why you are confused now and what I missed that has caused you that confusion. When you said this...

I am not clear really as to why point 2 is not already covered off by the SSL that has already been established?

.. I didn't realise that you were talking about "the SSL that has already been established" in your code that you are working on. I thought you were just talking about SSL in general still.

So... I'll explain where I was going with all that..

- If you only use the code yourself, or if you only share it privately between a limited number of people, then YES the code as it is covers off points 1 AND 2.
- If you then go and make your code publicly available to the general population, then that is where it now only covers off point 1 (the reason is that making the code public, makes the private key available to the public and that is what allows the active eavesdropper in point 2 to be able to potentially intercept and decrypt anyone else's traffic that is using your code.
- If you then implement the ideas that I was suggesting, where you have the server code generate it's own key pair, then you are again covering off point 2, because now the private key is different for every client/server connection and so an attacker can't listen in to other's conversation.

Sorry for the confusion, but does that make things clearer now?



I want to understand whether the data *whilst in transit* is encrypted

So yes, the data is always encrypted! And that is why I say that all these ways to implement SSL covers off point 1, because you can't just look at the traffic going between users and see the data in plain text. Now I know that you say that you aren't totally interested in all the details just yet (and that's fine) but another way of putting the point that I am trying to make is, "even if the data in transit is encrypted, if the key that decrypts that data is easily available, there is not a lot of difference between that encrypted data and the actual plain text it represents"


However, now reading the rest of your post and in particular the 2 points that you make, my suggestion would be to not use SSL at all !!! ;-) Honestly, using SSL is not really providing you with much benefit and is definitely throwing is a lot of complication for that little benefit. What *I* would do in your case, is just open plain socket connections but implement a simple symmetric encryption with the key derived from your USB stick ID. This would meet both points that you mention, and I believe would be fair simpler than the SSL based code that you already have considering you still need to add further code to authenticate on the USB ID.
Yes, that makes the situation very clear and sorry to have mucked you around to explain it all again, but it is all crystal now.

I would like to go the server-generated SSL route, but maybe after getting the enhanced authentication (based on the usb stick handle) idea done? Could I get back to you on the server-generated cert (could be called SGCert for short ;) ), at some point in the nearish future?
I'm glad that is all clear now!

Actually, I think I have an even better/easier way to go about this. That is basically to use pretty much the code that you have now, i.e. NOT generating the key for each installation, but just using the same key everywhere. BUT, in order to protect that key, instead of using the hardcoded "key password" that you have (currently lssetna), just use the USB Stick ID as the key password (and the keystore password). This way, you don't have to hardcode the password in to your code, meaning that even if you do distribute the code (which includes the keystore) publicly, no one can get to the private key without knowing that USB stick id. Best of both worlds, no one (who doesn't know the USB id) can get the private key and therefore can NOT eavesdrop on others conversations, and also can't make connections with others without having that password. And it is easy to implement, as I said just use your current code, except just replace the hard-coded "lssetna" with the value that you retrieve from the registry. Finally, you also just have to regenerate your keystore using keytool but provide the USB stick id when asked for the password. Done!
That's quite a comment, and has considerable potential merit if I am not mistaken. However, what about the 'client' though ? Given that (minimally) there would be only two people in the conversation, hopefully they would not both need the "same" stick ? I had in mind that the 'client' (which, lol, means BOTH parties) would have access to the stick id only via it being included in the source as a hardcoded var . . . . ? Obviously two or more parties cannot share the same stick. See my point ?

IOW the 'server' side would of course have access to the stick's id via the registry, but the 'client' would not have such access.

So I guess if the stick holder wished to connect out (iow, to be the ‘client’), the stickless ‘server’ side would need to receive the key from the stick-side’s registry first. Whereas if the stickless side was the client, then it would be a bit more straightforward.
Given that (minimally) there would be only two people in the conversation, hopefully they would not both need the "same" stick ?

I do see your point, but it has just made it obvious that *I* had completely missed your point earlier ;-)

So... I'm happy if you just want to go through what we have talked about here and try some things out, etc. But if you then want some more help, can I ask a favour? In order for me to fully get my head around what you are trying to do, can you maybe give a more high-level description of this application, i.e. how you think it would be distributed and to who, why they would want to use it, and a high level description of how they would go about it, etc.
made it obvious that *I* had completely missed your point earlier ;-)

Oow - well I had no inkling from my side that you had missed any point(s) at all . . . until you said that, I saw no indication that our wires were crossed (apart from my excursion into asking you for a re-explanation of the famous "point 2" situation. ; ) If you did miss some point I was making, it was more than likely because I hadn't explained it well.

So if you'd care to tell me what the point was again, I can then tell you whether I think you missed it or not - if you get my drift . . . . .  ?? ; )

As to the bigger implementation picture I'd be happy to explain that as you've requested, but maybe if you'd get back on this 'missed point' point before I do that, it might clear up the source of an otherwise compoundable problem . . . ?

Prior to explaining exactly what I am aiming at, this could be a close relative :

Briar

:)
mccarl - can I ask you - what strength of encryption is being used by the code we've discussed above ? I don't remember that ever being set or settable in anything I've seen or that we discussed .  : )

Thnx,
k