Why http proxy C++ code works for http site but NOT works for https/ssl site

Dear Experts,

In last week, I have studied http proxy &  get simple example C++ code from internet
&  run it  on cmd.exe shell and I set the proxy setting on my firefox browser to
localhost and port 8080 for the C++ proxy program.  When I type  http://mylinuxsite.com on Firefox,  the C++ program will echo out the  site header info such as follows  & then it  will send the header info as request  to the remote server(http://mylinuxsite.com)  & get the repsonse, and  send back to firefox browser through the program. So far so good.  The C++ code is working completely  & successfully.

Now I try to  do the same for https/ssl site such as https://mylinuxsite.com, the same program is just looping sending header info to the server , the https/ssl header info is different attacted as follow (from previous http site),and no HTTPS response from the server. Why ? besides https://mylinux.com, all other https/ssl site ALSO  not working  through my proxy C++ program BUT works without proxy
on browser.

I have checked the first words of header is "CONNECT" that is for SSL conection
but I found one setting that is "Proxy-Connection: keep-alive",what is that ?
whether I could delete it & send the https header  to server again that will work ?

Any suggestion or reason Why the program can NOT work   for https/SSL site access ?
Be reminded: https//mylinux.com is working on Firefox browswer WIHTOUT proxy
after testing since I have already installed openSSL on my linux server

Question-2: I've checked  the http method at http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html for "CONNECT" option,  If https/ssl site need to do POST form request,
how to do that to set  POST method option on header with "CONNECT" method together
for https site access  ?

Hope you understand what I am asking, if not please pt it out

Duncan
 
Firefox header info for normal http
=============================
GET http://mylinuxsite.com/b.php HTTP/1.1
Host: mylinuxsite.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive

Open in new window


Firefox header info for HTTPS/SSL
=============================
CONNECT mylinuxsite.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: mylinuxsite.com

Open in new window

LVL 13
duncanb7Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jkrCommented:
A plain HTTP proxy can't handle the SSL handshake - so the connection will be dropped before it is established.
0
duncanb7Author Commented:
Dear jkr, could you talk more about the concept of how it works for SSL proxy ?
or what is major different between http proxy and https/ssl proxy ?


After searching SSL proxy in internet, now I know the code for SSL proxy is much more
complicated than HTTP proxy.

But  I just dump the Firefox's request on my http proxy to the remote server on behalf of
browser role, at least the remote  server will send back request error message to  client
,but it didn't. Whether it is related request such as "CONNECT.....HOST:....." that
 is NOT encrypted by Firefox browser or by my http proxy  so that the remote server openSSL decrypt the request in wrong format ? If so, I need to do ecnrypt  it according
to the SSL cert , key in order to connect to remote server, Right ?

Normally I know Firefox will do encryption for SSL site if there is NO proxy, and
firefox will just dump request in text to http proxy if proxy is enabled. Is it correct ?

Please advise

Duncan
0
jkrCommented:
OK, let me try - a HTTPS proxy would have to act mainly as a tunneling server, i.e exchanging both the handshake protocol and the ensueing communication seamlessly. Or act as a 'man in thew´middle' (yet the idea would be flawed IMO). This is due to the SSL/HTTPS protocol (http://en.wikipedia.org/wiki/HTTP_Secure), where keys have to be exchanged during the connection setup. A HTTP Proxy can't do that. The 'proxy in the middle' approach as above requires the proxy to do exactly that, which is a concept that breaks the rules uf trust in SSL connections and I therefore would not recommend to use it (http://en.wikipedia.org/wiki/Proxy_server).
0
Acronis True Image 2019 just released!

Create a reliable backup. Make sure you always have dependable copies of your data so you can restore your entire system or individual files.

duncanb7Author Commented:
Dear jkr, thanks for your reply,

You mean HTTPS proxy acts as tunneling server that is similar to SSH tunneling we are using from Putty software on home PC computer for remote server shell access , Is it correct ?

Could you talk more a litte bit and reveiw my question in first post on this thread
and answer it in more detail ?

Please advise

Duncan
0
jkrCommented:
Well, 'in more detail' and 'encryption' are a bit controversary, but OK: Yes, it is not only similar to SSH tunneling, it is quite like that. I guess my main point is that you don't want to use proxies here. As for the details, what exactly are you inerested in?
0
c_kedarCommented:
Key part of SSL protocol is that client (browser) should be able to talk to server 'privately', i.e. the conversation can not be eves-dropped by anybody on the way.
This is accomplish this, as part of SSL handshake,
browser creates a symmetric key for encryption/decryption, encrypts it using public key found in certificate sent by server, and sends it to server
this can be decrypted only with private key which is available with only with server
browser expects a response from server encrypted with the key it had sent, to kind of confirm that it is talking directly to right server.

When proxy server comes in middle, it should not be able to see content of http protocol being transacted by client and server.
For this to happen it has to act as simple tunnel, i.e. forwarder of bytes.

The sample you have complied may not have this functionality and hence it is not working.
0
duncanb7Author Commented:
dear c_kedar,
When proxy server comes in middle, it should not be able to see content of http protocol being transacted by client and server.
For this to happen it has to act as simple tunnel, i.e. forwarder of bytes.

You mean proxy server that is refered to HTTPS/SSL proxy server and  NOT http proxy sever Right ?

Before doing what you say like key or encryption stuff in your last post, the first client request is "CONNECT" request,  right ? I just send the "CoNNECT" request string to server and that string is  sent from Firefox browser and received by my http proxy program

when I send this

"CONNECT mylinuxsite.com:443 HTTP/1.1\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0\r\n
Proxy-Connection: keep-alive\r\n
Connection: keep-alive\r\n
Host: mylinuxsite.com\r\n\r\n"

to the serve, the server should repsonse my "CONNECT" , at least some http error message  to client. Why there is nothing ?

 Please advise
Duncan
0
c_kedarCommented:
the server should repsonse my "CONNECT" , at least some http error message  to client. Why there is nothing ?

The server code might be silently ignoring CONNECT method (or possibly any method it has not implemented). Really depends on the server implementation. Search in the code to confirm.

I noticed you had second question in your original post.
You can not/do not combine POST with CONNECT.
The way it works is CONNECT method is request from browser to proxy server to establish a tunnel to reach remote server. Then browser will send whatever HTTP request it wants to do, GET/POST/PUT/OPTION etc, directly to remote server.
0
duncanb7Author Commented:
Dear  c_kedar

The server code might be silently ignoring CONNECT method (or possibly any method it has not implemented). Really depends on the server implementation. Search in the code to confirm.

if it is "CONNECT" method issue on my remote linux server, why the broswer will send
the "CONNECT" to the proxy program ? Is it because browser is asking the proxy program
to do "CONNECT" http tunnel connection, and then perform https/SSL request.

For normal cases or mode, I mean no proxy server help for browser, the browser
will perform HTTP/SSL request directly to the remoter server without any http tunnel
"CONNECT" connection help.

The resson why my proxy  program that is not working for "connect" that is because
the simple proxy program didn't have such code to do  http tunnel "CONNECT" connection.
And that is all totally  NOT related to remoter server

For "CONNECT" method  implementation, it can be done by Putty's SSH http tunnel(forward port) + Firefox browser proxy setting for SOCK v5  , I tested it before
and it works for all http and https request. Just check my concept to "CONNECT" method
right or not .

Please advise my question and pt out what I said that is correct or not.

Duncan
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
duncanb7Author Commented:
Thanks for all of your reply

Have a nice day

Duncan
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
SSL / HTTPS

From novice to tech pro — start learning today.