venks
asked on
How to Create Index For web page?
Can some one give me code for reading a web site & generating an Index page giving all the links listed ion the web page.For example if there is a web site giving 5 different categories & with each category having many individual html pages ,h ow do i get the links of these individual html files in an Index file?
Venkatesh
Venkatesh
ASKER
Hello Andrew
Thanks for your answer.How ever i want the code to do the task at hand
Venkatesh
Thanks for your answer.How ever i want the code to do the task at hand
Venkatesh
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I don't have such code at hand, but here's how you should be doing it:
- Navigate to the desired main page
- Using RegExps, extract all links in the page, no matter what.
- Run several threads that navigate to the links and gather all the links.
Now, you'll have a series of links, maybe driven by conditions you set in your program.
Your options at this point may be the most varying, you could just put them in a database and index them this way or
keyword-based, it's up to you. If you know the website and don't need a too general rule, you could just filter your
links accordingly.
HTH,
Andrew