Solved

how to navigate website using Jsoup in java

Posted on 2013-06-11
6
1,553 Views
Last Modified: 2013-06-25
I need help I am learning Jsoup and I need to know how can i navigate in Jsoup to a different link, for this example I have done the basic get the title, get links and get texts. But I want to be able to use one of those child links and go to the inside that child link. For example from google web page I want to be able to go to youtube page because its one of the child links in google and once in youtube pick another child link and than be able to grab a string. How would I be able to do this in Jsoup? Thanks in advance!

import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;


public class JSoupTest {


public static void main(String args[]) {

    try {


        Document doc=Jsoup.connect("http://www.google.com").get();

        // get page title
        String title = doc.title();
        System.out.println(title);

        //gets all links
        Elements links = doc.select("a[href]");
        for (Element link : links) {

        // get the value from href attribute
        System.out.println("\nlink : " + link.attr("href"));

        }

        for( Element element : doc.select("p") )    
                    // Select all 'p'-Tags and loop over them
        {
            if( element.hasText() )                 
                    // Check if the element has text (since there are some empty too)
            {
              System.out.println(element.text()); // print the element's text
            }
        }


    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

Open in new window

0
Comment
Question by:yescobar2012
  • 4
  • 2
6 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 39238538
How will JSoup help you if some of the links are generated by javascript?
0
 

Author Comment

by:yescobar2012
ID: 39238847
But I have seen similar examples that does that using Jsoup, even though that source is in javascript
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39238944
JSoup supports javascript?? I didn't think so ...
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:yescobar2012
ID: 39239285
If you can extract the links from the source you can use them.
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 500 total points
ID: 39239454
I'm talking about javascript running in memory and therefore the DOM, not just links as text in the page
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39275455
:)
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Suggested Solutions

Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
Use these top 10 tips to master the art of email signature design. Create an email signature design that will easily wow recipients, promote your brand and highlight your professionalism.
In this tutorial viewers will learn how to embed an audio file in a webpage using HTML5. Ensure your DOCTYPE declaration is set to HTML5: : The declaration should display (CODE) HTML5 is supported by the most recent versions of all major browsers…
The viewer will the learn the benefit of plain text editors and code an HTML5 based template for use in further tutorials.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now