Solved

How to design complicated program with Supervisor/worker/threads?

Posted on 2004-04-19
6
306 Views
Last Modified: 2010-03-31
I'm really really lost right now in class.  I understand the basic concept of the Supervisor/worker pattern of developing a program, but have no idea how to design the following program.  My professor isn't much help, as I have asked him numerous questions with no results.  The problem is that I haven't had any networking/internet socket whatever you want to call it kind of class yet.  So the following program to be designed is a nightmare for me.  Please Help!  I must...

Develop a supervisor-worker system that will retrieve information from several web sites in parallel. The supervisor should provide a function to the outside world:
    public void getSites( int num_workers, ArrayList sitelist, ArrayList sitedata )
The caller will provide a list of URL's in sitelist. The entries will be structured as follows:  element i will be a computer name and element i+1 will be the item to retrieve. So, for example, if the user wants to retrieve these three web pages:
    http://www.cnn.com/index.html
    http://www.wired.com/news/
    http://www.lwn.net/Articles/

The sitelist list will contain these strings:
    "www.cnn.com"
    "/index.html"
    "www.wired.com"
    "/news/"
    "www.lwn.net"
    "/Articles/"

When the getSites() function returns, the array sitedata should have the results from each site (as strings), in order (so sitedata.get(0) would return the data from cnn.com; sitedata.get(1) returns the data from wired, and so forth).

After sending this data, the server responds with the contents of the site, including a few headers. You can include these in your sitedata ArrayList along with the rest of the site's response. HTTP requests are made to port 80.

How you structure your supervisor and workers is up to you, but each one should be its own object. The caller may want several workers to work simultaneously, so your program should work correctly for any number requested (it will be at least one, but your code should not impose an upper limit, in case the user has a really fast computer and wants to stress-test it).

You will probably want to create some test program that will call the supervisor with a list of sites to check. If you want to get fancy (which I don't), you can make a GUI interface with progress reporting, thermometers, and whatever else you like.
0
Comment
Question by:tmp09
  • 4
6 Comments
 
LVL 92

Expert Comment

by:objects
ID: 10865469
you can use the URLConnection class to download page from varioius sites.
0
 
LVL 92

Accepted Solution

by:
objects earned 500 total points
ID: 10865510
your worker would look something like:

public class Worker implements Runnable
{
   private Supervisor Parent = null;

   public Worker(Supervisor parent)
   {
      Parent = parent;
   }

   public void run()
   {
      while supervisor still has pages to download
      {
          get next page to download from supervisor
          download page using URLConnection
          pass content back to supervisor
      }
   }
}
     
0
 
LVL 92

Expert Comment

by:objects
ID: 10865517
your supervisor would start the required number of worker threads and then wait until they have completed downloading all required pages.
0
 
LVL 92

Expert Comment

by:objects
ID: 10865525
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to parse custom JSON to POJO java 4 72
Where to store the queries for modification of table 4 61
use lov values 2 49
hibernate example for saving data 19 37
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now