Sending a Url and receiving

I want to write a Java program which opens a socket to send a URL and gets the response - i.e. contents of web site
which I can then Output

can anyone show me i quick example of how to do this?

plaskowjAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

vladi21Commented:
0
vladi21Commented:
0
plaskowjAuthor Commented:
This great, but I am still looking for a simple bit of code -

I will make it worth your while.

All it has to do is send a URL and receive the output-


please help
0
Cloud Class® Course: Microsoft Windows 7 Basic

This introductory course to Windows 7 environment will teach you about working with the Windows operating system. You will learn about basic functions including start menu; the desktop; managing files, folders, and libraries.

Ravindra76Commented:

Hi,

Now this program prints to console.
YOu can write to file if you want

import java.io.*;
import java.net.*;

public class GetInfo {

      public static void main(String args[]) throws Exception {

            URL url = new URL("http://www.yahoo.com");
            InputStream in = url.openStream();
            BufferedReader br = new BufferedReader(new InputStreamReader(in));
            while ( true ) {
                  String line = br.readLine();
                  if ( line ==null ) break;
                  System.out.println(line);
            }

      }


}
0
sen_kumCommented:
import java.net.* ;
import java.io.* ;
class Test
{
  public static void main( String args[] ) throws Exception
  {
    String aLine,content ;
    aLine = "" ;
    content = "" ;
    URL url = new URL( "www.urlString.com/index.htm" ) ;
    InputStream in = url.openStream() ;
    BufferedReader dis = new BufferedReader(new InputStreamReader(in));
    while ( ( aLine = dis.readLine() ) != null )
    {
      content = content + aLine ;
    }
    System.out.println( content ) ;
  }
}
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
plaskowjAuthor Commented:
Seeing that you answered that,

can you also tell me with a brief example if a get Socket to listen on a port, is there anyway of determining the url requested by the User?
0
vladi21Commented:
its from http://tactika.com/realhome/javaht/java-f1.html

So where my points? :)

import java.io.*;
 import java.net.*;

 public class suckURL {
   String aFile;
   String aURL;

   public static void main(String args[]) {
     // GIF  JAVA How-to  at Real's Home
     String url =
       "http://www.tactika.com/realhome/images/";
     suckURL b = new suckURL(url, "jht.gif");
     b.doit();
     }

   suckURL(String u, String s){
     aURL  = u;
     aFile = s;
     }

   public void doit() {
     DataInputStream di = null;
     FileOutputStream fo = null;
     byte [] b = new byte[1];  
       
     try {
       System.out.println("Sucking " + aFile);
       System.out.println("   at " + aURL );
       // input
       URL url = new URL(aURL + aFile);
       URLConnection urlConnection = url.openConnection();
       urlConnection.connect();
       di = new DataInputStream(urlConnection.getInputStream());

       // output
       fo = new FileOutputStream(aFile);

       //  copy the actual file
       //   (it would better to use a buffer bigger than this)
       while(-1 != di.read(b,0,1))
         fo.write(b,0,1);
         di.close();  
         fo.close();                
         }
       catch (Exception ex) {
         System.out.println("Oups!!!");
         ex.printStackTrace();
         System.exit(1);
         }
       System.out.println("done.");  
     }
 
0
plaskowjAuthor Commented:
Seeing that you answered that,

can you also tell me with a brief example if a get Socket to listen on a port, is there anyway of determining the url requested by the User?
0
plaskowjAuthor Commented:
I'll send you loads of points if you answer my last comment - cheers
0
vladi21Commented:
U mean server socket?
You must implement some protocol
Maybe u want to write http server?
0
plaskowjAuthor Commented:
Yes, can you give me pointers,  what i am implmenting an application that has a thread listening ona socket.

Anyhow i want to read off the socket - and retireve the URL requested.

Can you help?
0
vladi21Commented:
better use servlets

What are servlets?

  Servlets are server-side Java applications, as opposed to
  client-side applets or standalone applications. While
  servlets are compatible with many different types of servers,
  typically they are used in web servers, as a replacement for
  CGI scripts or Active-Server Pages (ASP).

  Java servlets offer many advantages over other forms of
  server-side processing. Apart from the obvious (they are
  written in the Java programming language, a big plus after
  all), servlet based applications are far easier to write
  than CGI scripts. There's no need to write code for parsing
  HTTP request parameters, as this code is provided by the
  javax.servlet.http package. You have access to the entire
  Java API, with no networking restrictions (making servlets
  far more attractive than applets). There are also performance
  increases over CGI scripts, as servlets persist over time,
  and do not create a new process for every connection.

http://www.as400.ibm.com/developer/java/documents/servlet1.html


Purple Servlet FAQ  http://www.purpletech.com/java/servlet-faq/
http://www.newatlanta.com/servlets/servlet-faq.html 
http://www.dmoz.org/Computers/Programming/Languages/Java/Runtime_Environments/Servlets/faq.html

The Servlet home page on the Javasoft website
http://java.sun.com/products/java-server/servlets/
http://www.coolservlets.com/

Servlet Essentials by Stefan Zeiger
http://www.novocode.com/doc/servlet-essentials/

Paul Flavin's Servlet Resources page
http://www.frontiernet.net/~imaging/servlets_intro.html
http://oop.rosweb.ru/cetus/oo_java_servlets.html
http://dmoz.org/Computers/Programming/Languages/Java/Runtime_Environments/Servlets/


servlets and JSP
http://www.apl.jhu.edu/~hall/java/Servlet-Tutorial/

Servlet Central, the Server-Side Java Online Magazine
http://www.servletcentral.com 

A list of Servlet engines and Servlet-enabled Web Servers
http://jserv.javasoft.com/products/java-server/servlets/environments.html

http://www.davidreilly.com/java/java_network_programming/jnpfaq.txt
6.1 What are Java Servlets?
  6.2 What do I need to develop servlets?
  6.3 Where can I get more information on servlets?
  6.4 How does servlet performance compare to applets?
  6.5 How does servlet performance compare to CGI?
  6.6 Should I use single-threaded, or multi-threaded
      servlets?
  6.7 How do I send cookies from a servlet?
  6.8 How do I read browser cookies from a servlet?
  6.9 How do I make cookies expire after a set time period?
  6.10 Why aren't cookies stored by my servlets accessible
       to my CGI scripts or ASP pages?
  6.11 How can I void a cookie, and delete it from the
       browser?
0
plaskowjAuthor Commented:
I don't want to use servlets, as my application is a proxy server - where a gateway will route all URL's entering a specific port to my application.

So therefore i have to keep get the URL from a socket - please help
0
vladi21Commented:
HTTPServer

import java.awt.*;
import java.net.*;
import java.io.*;
import java.util.*;

public class HTTPServer extends Thread {
      public final static int DEFAULT_PORT = 80;
      protected int port;
      protected ServerSocket server;
      protected ThreadGroup currentConnections;
      protected TextArea logs;
      protected Vector connections;

      public static void fail(Exception e, String msg) {
            System.err.println(msg + ": " + e);
            System.exit(1);
      }

      public HTTPServer(int port) {
            super("HTTPServer");
            if (port == 0) port = DEFAULT_PORT;
            this.port = port;
            try {
                  server = new ServerSocket(port);
            }
            catch (IOException e) {
                  fail(e, "Exception creating server socket");
            }
            currentConnections = new ThreadGroup("Server Connections");
            Frame f = new Frame("Server Status");
            logs = new TextArea();
            logs.setEditable(false);
            f.add("Center", logs);
            f.resize(400, 200);
            f.show();
            connections = new Vector();
            this.start();
      }

      public void run() {
            try {
                  while (true) {
                        Socket ClientSocket = server.accept();
                        Connection c = new Connection(ClientSocket, currentConnections, 3, logs);
                        synchronized(connections) {
                              connections.addElement(c);
                              logs.appendText(c.getInfo() + "\n");
                        }
                  }
            }
            catch (IOException e) {
                  fail(e, "Exception while listening for connections");
            }
      }

      public static void main(String args[]) {
            int port = 0;
            if (args.length == 1) {
                  try {
                        port = Integer.parseInt(args[0]);
                  }
                  catch (NumberFormatException e) {
                        port = 0;
                  }
            }
            new HTTPServer(port);
      }
}

class Connection extends Thread {
      static int numberOfConnections = 0;
      protected Socket client;
      protected DataInputStream in;
      protected PrintStream out;
      private TextArea logs;

      public Connection(Socket clientsocket, ThreadGroup currentconnections, int priority, TextArea logs) {
            super(currentconnections, "Connection number" + numberOfConnections++);
            this.setPriority(priority);
            client = clientsocket;
            try {
                  in = new DataInputStream(client.getInputStream());
                  out = new PrintStream(client.getOutputStream());
            }
            catch (IOException e) {
                  try {
                        client.close();
                  }
                  catch (IOException e2) {}
                  System.err.println("Exception while getting socket streams: " + e);
                  return;
            }
            this.logs = logs;
            this.start();
      }

      public void run() {
            String inline;
            String header, request = "";
            String currentdir = System.getProperty("user.dir");
            try {
                  while(true) {
                        inline = in.readLine();
                        request += inline + "\n";
                        if (inline.length() == 0) {
                              logs.appendText(request + "\n");
                              StringTokenizer parse = new StringTokenizer(request);
                              if ((parse.nextToken()).equalsIgnoreCase("GET")) {
                                    inline = parse.nextToken();
                                    FileInputStream f = null;
                                    try {
                                          f = new FileInputStream(currentdir + inline);
                                    }
                                    catch (FileNotFoundException e) {
                                          out.println("File " + inline + " not found");
                                          break;
                                    }
                                    header  = "HTTP/1.1 200 Document follows\n";
                                    header += "Date: " + (new Date()).toGMTString() + "\n";
                                    header += "Server: PlaQ Server\n";
                                    File info = new File(currentdir + inline);
                                    header += "Last-modified: " + (new Date(info.lastModified())).toGMTString() + "\n";
                                    header += "Content-type: ";
                                    String name = info.getName();
                                    logs.appendText("File: " + name + "\n");
                                    int idx = name.indexOf(".");
                                    name = name.substring(idx + 1);
                                    if (name.equalsIgnoreCase("html") || name.equalsIgnoreCase("htm"))
                                          header += "text/html\n";
                                    else if (name.equalsIgnoreCase("jpeg") || name.equalsIgnoreCase("jpg"))
                                          header += "image/jpeg\n";
                                    else if (name.equalsIgnoreCase("gif"))
                                          header += "image/gif\n";
                                    else
                                          header += "text/plain\n";
                                    header += "Content-length: " + info.length() + "\n";
                                    header += " ";
                                    logs.appendText(header);
                                    out.println(header);
                                    out.flush();
                                    byte[] b = new byte[4096];
                                    int len;
                                    long bytestrasnfered = 0;
                                    try {
                                          while ((len = f.read(b)) != -1) {
                                                bytestrasnfered += len;
                                                out.write(b, 0, len);
                                          }
                                    }
                                    catch (IOException e) {}
                                    f.close();
                                    out.flush();
                                    logs.appendText("Transfered " + bytestrasnfered + " bytes\n");
                                    request = "";
                              }
                              else {
                                    out.println("Unknown command");
                              }
                        }
                  }
                  logs.appendText("End of while\n");
            }
            catch (IOException e) {}
            finally {
                  try {
                        logs.appendText("Close " + getInfo() + "\n");
                        client.close();
                  }
                  catch (IOException e2) {}
            }
      }

      public String getInfo() {
            return ("Connection from: " + client.getInetAddress().getHostName());
      }
}


----
Is your application  a  HTTP proxy server ?
0
vladi21Commented:
look
http://www.experts-exchange.com/jsp/qShow.jsp?ta=browsers&qid=10166511 

Just parse first string of HTTP request

example in Java:

if ( frame.useProxyCheckBox.getState() )
{
remoteHost=frame.proxyHost;
remotePort=frame.proxyPort;
}else
{
int second;
int first=request.indexOf(' ');
if (first>0)
{
second=request.indexOf(' ', first+1);
if (second>0)
{
//System.out.println("\n\nrequestURL="+request.substring(first+1,second)+"_____");
URL requestURL=new URL(request.substring(first+1,second));
remoteHost=requestURL.getHost();
remotePort=requestURL.getPort();
if (remotePort==-1) remotePort=80;
System.out.println("path="+requestURL.getFile());

request=request.substring(0,first+1)
+requestURL.getFile()
+request.substring(second);

}
//zzzzz
}
}


0
vladi21Commented:
if u want I can mail to you my sources of simple proxy server

Post new Q with lot of points and ur e-mail :)
0
Ravindra76Commented:

I did n't find any difference between my comment and proposed answer.
0
plaskowjAuthor Commented:
vladi 21-

I can't seem to browse to this through an Internet browser

why?

can you help?
0
vladi21Commented:
plaskowj : what a problem?
0
plaskowjAuthor Commented:
can you help?

i can set up a Client java application which can access this port, but i can't do it through a browser-

please help
0
vladi21Commented:
which port?

post new Q with description
and where my points ? :)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java EE

From novice to tech pro — start learning today.