how to navigate website using Jsoup in java

Posted on 2013-06-11
Last Modified: 2013-06-25
I need help I am learning Jsoup and I need to know how can i navigate in Jsoup to a different link, for this example I have done the basic get the title, get links and get texts. But I want to be able to use one of those child links and go to the inside that child link. For example from google web page I want to be able to go to youtube page because its one of the child links in google and once in youtube pick another child link and than be able to grab a string. How would I be able to do this in Jsoup? Thanks in advance!

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

public class JSoupTest {

public static void main(String args[]) {

    try {

        Document doc=Jsoup.connect("").get();

        // get page title
        String title = doc.title();

        //gets all links
        Elements links ="a[href]");
        for (Element link : links) {

        // get the value from href attribute
        System.out.println("\nlink : " + link.attr("href"));


        for( Element element :"p") )    
                    // Select all 'p'-Tags and loop over them
            if( element.hasText() )                 
                    // Check if the element has text (since there are some empty too)
              System.out.println(element.text()); // print the element's text

    } catch (IOException e) {
        // TODO Auto-generated catch block

Open in new window

Question by:yescobar2012
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
LVL 86

Expert Comment

ID: 39238538
How will JSoup help you if some of the links are generated by javascript?

Author Comment

ID: 39238847
But I have seen similar examples that does that using Jsoup, even though that source is in javascript
LVL 86

Expert Comment

ID: 39238944
JSoup supports javascript?? I didn't think so ...
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 39239285
If you can extract the links from the source you can use them.
LVL 86

Accepted Solution

CEHJ earned 500 total points
ID: 39239454
I'm talking about javascript running in memory and therefore the DOM, not just links as text in the page
LVL 86

Expert Comment

ID: 39275455

Featured Post

Instantly Create Instructional Tutorials

Contextual Guidance at the moment of need helps your employees adopt to new software or processes instantly. Boost knowledge retention and employee engagement step-by-step with one easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Java array 21 141
Strange router problem - can't access 14 103
iframe stay to the left of page 5 26
CSS styling problem 3 21
In this post we will learn different types of Android Layout and some basics of an Android App.
This article demonstrates how to create a simple responsive confirmation dialog with Ok and Cancel buttons using HTML, CSS, jQuery and Promises
In this tutorial viewers will learn how to embed an audio file in a webpage using HTML5. Ensure your DOCTYPE declaration is set to HTML5: : The declaration should display (CODE) HTML5 is supported by the most recent versions of all major browsers…
In this tutorial viewers will learn how to embed videos in a webpage using HTML5. Ensure your DOCTYPE declaration is set to HTML5: "<!DOCTYPE html>": Use the <video> tag to insert a video. Define the src as the URL of your video; this is similar to …
Suggested Courses

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question