Link to home
Start Free TrialLog in
Avatar of venks
venks

asked on

How to Create Index For web page?

Can some one give me code for reading a web site & generating an Index page giving all the links listed ion the web page.For example if there is a web site giving 5 different categories & with each category having many individual html  pages ,h ow do i get the links of these individual html files in an Index file?
Venkatesh
Avatar of DeerBear
DeerBear

Hi.

I don't have such code at hand, but here's how you should be doing it:

- Navigate to the desired main page
- Using RegExps, extract all links in the page, no matter what.
- Run several threads that navigate to the links and gather all the links.

Now, you'll have a series of links, maybe driven by conditions you set in your program.

Your options at this point may be the most varying, you could just put them in a database and index them this way or
keyword-based, it's up to you. If you know the website and don't need a too general rule, you could just filter your
links accordingly.

HTH,

Andrew
Avatar of venks

ASKER

Hello Andrew
Thanks for your answer.How ever i want the code to  do the task at hand
Venkatesh
ASKER CERTIFIED SOLUTION
Avatar of DragonSlayer
DragonSlayer
Flag of Malaysia image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial