?
Solved

Connection to Yahoo

Posted on 2003-04-01
4
Medium Priority
?
231 Views
Last Modified: 2010-03-31

How can i manage to connect to Yahoo send a query and get the answers back? is there a way to avoid parsing the HTML by using an existing API.. or by using a little trick?


thanks in advance.
0
Comment
Question by:Caillou
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 3

Expert Comment

by:msterjev
ID: 8248202
Use a query :

http://search.yahoo.com/bin/search?p=Java

and HttpURLConnection:

What do you mean by avoiding parsing the HTML? What do you want to retrieve?
0
 
LVL 4

Accepted Solution

by:
ShannonE earned 200 total points
ID: 8248225
Hey man,
The HTML from commercial sites is usually WAY too complicated to parse (thats why XML is so popular ;). You could try Java regular expressions, but chances are your reg expressions for yahoo won't work for some other search site, and even Yahoo might change the structure of its HTML months down the road. But you can do it if you bypass parseing the document and just show it to the user by opening up the default browser from your Java program and loading the results page.

Heres the code:
//-------------------------
//file BrowserControl.java
//-------------------------
import java.io.*;

public class BrowserControl
{
    /**
     * Display a file in the system browser.  If you want to display a
     * file, you must include the absolute path name.
     *
     * @param url the file's url (the url must start with either "http://"
or
     * "file://").
     */
    public static void displayURL(String url)
    {
        boolean windows = isWindowsPlatform();
        String cmd = null;
        try
        {
            if (windows)
            {
                // cmd = 'rundll32 url.dll,FileProtocolHandler http://...'
                cmd = WIN_PATH + " " + WIN_FLAG + " " + url;
                Process p = Runtime.getRuntime().exec(cmd);
            }
            else
            {
                // Under Unix, Netscape has to be running for the "-remote"
                // command to work.  So, we try sending the command and
                // check for an exit value.  If the exit command is 0,
                // it worked, otherwise we need to start the browser.
                // cmd = 'netscape -remote openURL(http://www.javaworld.com)'
                cmd = UNIX_PATH + " " + UNIX_FLAG + "(" + url + ")";
                Process p = Runtime.getRuntime().exec(cmd);
                try
                {
                    // wait for exit code -- if it's 0, command worked,
                    // otherwise we need to start the browser up.
                    int exitCode = p.waitFor();
                    if (exitCode != 0)
                    {
                        // Command failed, start up the browser
                        // cmd = 'netscape http://www.javaworld.com'
                        cmd = UNIX_PATH + " "  + url;
                        p = Runtime.getRuntime().exec(cmd);
                    }
                }
                catch(InterruptedException x)
                {
                    System.err.println("Error bringing up browser, cmd='" +
                                       cmd + "'");
                    System.err.println("Caught: " + x);
                }
            }
        }
        catch(IOException x)
        {
            // couldn't exec browser
            System.err.println("Could not invoke browser, command=" + cmd);
            System.err.println("Caught: " + x);
        }
    }
    /**
     * Try to determine whether this application is running under Windows
     * or some other platform by examing the "os.name" property.
     *
     * @return true if this application is running under a Windows OS
     */
    public static boolean isWindowsPlatform()
    {
        String os = System.getProperty("os.name");
        if ( os != null && os.startsWith(WIN_ID))
            return true;
        else
            return false;

    }
    /**
     * Simple example.
     */
    public static void main(String[] args)
    {
        displayURL("http://www.javaworld.com");
    }
    // Used to identify the windows platform.
    private static final String WIN_ID = "Windows";
    // The default system browser under windows.
    private static final String WIN_PATH = "rundll32";
    // The flag to display a url.
    private static final String WIN_FLAG = "url.dll,FileProtocolHandler";
    // The default browser under unix.
    private static final String UNIX_PATH = "netscape";
    // The flag to display a url.
    private static final String UNIX_FLAG = "-remote openURL";
}


//-------------------------------------
//File Yahoo.java
//-------------------------------------
import java.net.*;
import java.io.*;

public class Yahoo {

   public static void main (String args[]) {
         try {
         // make connection
               URL url = new URL("http://search.yahoo.com/bin/search?p=" +
                    URLEncoder.encode(args[0]));
        URLConnection connection = url.openConnection();
        connection.setDoInput(true);
        InputStream in = connection.getInputStream();

        // read reply
        StringBuffer b = new StringBuffer();
        BufferedReader r = new BufferedReader(new InputStreamReader(in));
        String line;
        while ((line = r.readLine()) != null) {
                  b.append(line);
            }

        String s = b.toString();
            // look for first search result, if any

        System.out.println("Entire string:\n" + s);
               PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("results.html")));
        out.print(s);
        out.close();
        BrowserControl bc = new BrowserControl();
        bc.displayURL("results.html");


    }

    catch (Exception e) { e.printStackTrace(); }

   }
}

To do your search, compile both files and type java Yahoo <search string>. Currently the search string comes from just the first command line arg value but you can easily change this.
Good luck!
0
 

Author Comment

by:Caillou
ID: 8253822
great Code Shannon,


I'm really impressed!

Actually i have to parse the HTML a little, 'cause i need to analyse the URLs returned by the query... I've manage to do so with AltaVista but i took me some time, that's why i meant by avoid parsing the HTML.

Anyway thanks a lot man.
0
 

Author Comment

by:Caillou
ID: 8253830
nothing to add... just excellent ;)
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Viewers learn about the scanner class in this video and are introduced to receiving user input for their programs. Additionally, objects, conditional statements, and loops are used to help reinforce the concepts. Introduce Scanner class: Importing…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.
Suggested Courses
Course of the Month13 days, 16 hours left to enroll

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question