Making an https connection

Forget that ;-)

What version of Java is Jetty running on?

Try using javax.net.ssl.HttpsURLConnection available from JDK 1.4 onwards, I think.

> What must I do/add to get this to work?

You need the proper key to connect to the server and from the error message you are getting it seems to me that the server does not recognize your client. In order to connect to a secure server (https) it is not only enough to just change the url from http to https but you also need a proper certificate. Make sure that you have a certificate that the server you are trying to connect to recognizes. If you do not know what this is get in touch with the people who provided you with the keystore.

Basically what is going on is that you are trying to exchange key information with the server and the information you send and the information that the server has does not match.

You can have a look at this link to get an idea of how things work: http://www.andrew.cmu.edu/~mm6/95-733/PowerPoint/SSL.ppt

CEHJ: Jetty is running on 1.4.2_06

praveen: As the URL.openConnection method will return an appropriate object, which in this case will be an HttpURLConnection object or something derived from it, and since I am only using methods defined in HttpURLConnection, and since the cast worked, I can see no reason for making the cast more specific.

girionis: I was not (rightly or wrongly) provided with a keystore. Doesn't some "automatic" mechanism exist to manage this stuff? When I go to look at a secure site with my web browser I don't have to first have the web owners "provide me with a keystore". The browser does some stuff, obtains a certificate (I assume this is something that contains the public key of the entity one is trying to connect to), holds it up to the browser user for inspection in a dialog, and then (if the user OK's it) everything proceeds.

Many of the common browser already implement support for https which allows access to secured communications without requiring the socket-level API provided with JSSE, but there are also cases where you are prompted with a pop up box that informs you if you want to accept the certificate.

Try this approach: http://javaalmanac.com/egs/javax.net.ssl/TrustAll.html and tell us if it makes any difference.

I always say this but use the jakarta http client.. it already has the SSL stuff figured out and will save you many headaches..
http://jakarta.apache.org/commons/httpclient/tutorial.html

imladris: the standard java.net.URL class doesn't support SSL. SSL involves encrypting the stream with PKCS, which involves a handshake to be established between the client and the server to exchange certificates/keys for the encryption. During the connection, the client and server exchange credentials and negotiate the security parameters.
You would have to install the public key certificate on the client(jetty webserver) of the host(server, "https://"+ipxml) you're establishing the connection with.
I totally agree with girionis's perception. the "automatic" mechanism exists for web browsers, which would prompt whether you want to install the certificate, but for programmatic JSSE connections, the certificate needs to be installed and valid before the connection can be made.

http://www.onjava.com/pub/a/onjava/2001/05/03/java_security.html looks like a pretty good introduction to me.

girionis: just completed the trustall approach with no apparent effect. I added the following to the end of the my jetty HttpHandler:

            TrustManager[] trustAllCerts=new TrustManager[]
            {      new X509TrustManager()
                  {      public java.security.cert.X509Certificate[] getAcceptedIssuers()
                        {      return(null);
                        }
                        public void checkClientTrusted(java.security.cert.X509Certificate[] certs,String authType)
                        {
                        }
                        public void checkServerTrusted(java.security.cert.X509Certificate[] certs,String authType)
                        {
                        }
                  }
            };
            
            try
            {      SSLContext sc=SSLContext.getInstance("SSL");
                  sc.init(null,trustAllCerts,new java.security.SecureRandom());
                  HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
            } catch(Exception e)
            {
            }


Also, I assume that what I would be losing with this approach is the assurance that I am talking to the correct host. I don't know whether that will be acceptable in the long run or not. I guess I'll try talking to the guys on the other side and see about getting a certificate from them.

Seriously try the jakarta client.. you will be running https in no time..

How would the jakarta client handle the situation where I don't have a certificate stored?

You don't need to do anything.. the httpclient acts just like a browser. Can you connect to their site using a browser?  If so using a java "browser" like httpclient will work in a similar fashion.  If you connect to their site and you get an untrusted popup asking you to trust they cert signer (e.g. it isn't verisign, thawte, etc), httpclient can handle these scenarios as well.

Investigation continues.

I have requested coding help from the guys on the other end; but they don't appear to see any problems with the code. It appears, in retrospect, that the other end is not an existing, performing, commercial site, yet, either. They listed a new ip and set of ports and requested I try those. Without the trustall code I got this exception: "javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found", which seems reasonable. With the trustall code I get: "javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?".

Does that suggest any useful action to take?

They will also be looking at the logs on their side.

Mathew, I will keep the httpClient in mind. I would prefer to not get involved in yet another license (no matter how mild) if I can avoid it. But I'll try it next week sometime if there isn't any progress.

The key here is to figure out which side the problem is on.. To do that I would use a process of elimination.. First things first.. which side is having the problem client or server?

To eliminate the server.. use a known client:
Have you tried accessing the site via a browser?  If so, what kind of warnings to you get?  Is the server cert signed by a known CA?

Next move down a layer to java:
Try the httpClient, is that it is a well tested and frequently used client and see if it works.. If it does and you don't want to use the apache license.. you can at least download the source code to see where your code isn't up to muster.

Investigation continues to continue.

Latest error I get back from the other side is:

java.io.IOException: HTTPS hostname wrong: should be <xxx.xxx.xxx.xxx>

between the angle brackets is the ip that I was using to reach it.
Girionis, does that suggest any useful action to take?


Mathew,

If I access the address with https://<ip>:port, then I first get the dialog indicating I am about to view pages over a secure connection, then I hit OK and it comes up with a Security Alert dialog asking me if I want to trust the certificate. If I proceed it then comes up with a dialog indicating there is both secure and nonsecure content on the page. When I indicate that YES I want to see the nonsecure items too, I get a page with an Error 403: Access forbidden; it is impossible to view this page. It is not accessible from this service.

I remain convinced that the standard Java tools ought to work. However, since I'm not getting anywhere, I took the plunge on setting up a parallel project with the Apache client. I won't bore with what it took to find the various pieces and figure out how to run them. Since the tutorial is for 3.0, I am running the 3.0 client which is from 3 days ago (11 Oct 2005) (Yikes!). Once I got it compiled it worked with http without any trouble.

However, setting the address to https got me the old

"HttpClient exception e: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found"

and I got that, even if I added girionis's TrustAll dodge. I assume, in retrospect, that code won't affect the internals of how the HttpClient operates. It's resetting something in HttpsURLConnection.

Here is the Apache version for the exchMsg method:

      String exchMsg(HttpResponse response,StringBuffer sb,boolean sendok)
      {      String postrsp=null;
            boolean scs=true;
            HttpClient client=new HttpClient();
            PostMethod post=new PostMethod("https://"+ipxml+":"+portxml);
            post.setRequestEntity(new StringRequestEntity(sb.toString()));
            try
            {      int retcode=client.executeMethod(post);
                  if(retcode!=HttpStatus.SC_OK)
                  {      Log.warning("error Posting: "+post.getStatusLine());
                        scs=false;
                  }
                  if(sendok)sendResp(response,"OK");
                  postrsp=post.getResponseBodyAsString();
            } catch(IOException e)
            {      Log.warning("HttpClient exception e: "+e);
                  scs=false;
            }
            if(scs)Log.event("RECV "+postrsp);
            else sendResp(response,"error");
            return(postrsp);
      }

What would you suggest as a next step?

OK, so much for this week. I'm off for the weekend. Back at it on Monday.

Ok here are your problems then. ...
>>asking me if I want to trust the certificate
and
>>Error 403: Access forbidden;

The first issue is that the CA that the server side folks used to sign their cert is not one of the few automagically trusted CAs.. they probably used a self-signed cert.  So you will need to import that certificate into your truststore before any java app will accept the cert.. this is analagous to you pushing OK from your browser.  
OR
using the apache http client you can use the EasySSLSocketProtocolFactory (http://jakarta.apache.org/commons/httpclient/sslguide.html) and have the client accept unknown CAs.  You probably don't want to do this in real production as it could be a security risk.. the site should get a real signed cert.

The 403 will give you problems no matter which client you attempt to use so that will need to be resolved on the server side to work properly.  You should be able to hit the server with your browser in any event without error.

Update: still working on this. Unfortunately there was a company move last week, so I didn't get anything done while the network was being taken down, moved, and set back up at the new location. I had to move to of course, and by the time I was up and running again, the company network was up, but then I had to get them to poke holes in the firewall again so I could continue testing this. Hopefully some new results in the next week or so.

Another update: I think I got a connection using girionis's dodges. I didn't get a sensible response though. Still working on it.

What was the response?

Something about not being able to parse.

They requested another attempt this morning, and this time I got the same response as I did on the HTTP link. So it looks like we're underway. I specifically attempted to remove both of girionis's dodges one at a time. But both are necessary to make this work.

Thanks girionis.

I explicitly raised the issue with them, that I am bypassing the checks on whether the software is connecting to the right host, but, so far, they seem unconcerned. I assume that the problem is that something along the following lines could occur: if someone could access the PC running this code and change the configuration to point it to another ip, they could fire up a program on that ip that receives the requests, records the information, passes the request to 3C, and passes back the response. That is, despite the fact that the traffic is encrypted, a single configuration change would enable someone to gain access to all the traffic from a particular site. Is that right? Is there something worse that you could think of happening?