How to Create Index For web page?

Posted on 2005-04-15
Last Modified: 2010-04-05
Can some one give me code for reading a web site & generating an Index page giving all the links listed ion the web page.For example if there is a web site giving 5 different categories & with each category having many individual html  pages ,h ow do i get the links of these individual html files in an Index file?
Question by:venks
    LVL 5

    Expert Comment


    I don't have such code at hand, but here's how you should be doing it:

    - Navigate to the desired main page
    - Using RegExps, extract all links in the page, no matter what.
    - Run several threads that navigate to the links and gather all the links.

    Now, you'll have a series of links, maybe driven by conditions you set in your program.

    Your options at this point may be the most varying, you could just put them in a database and index them this way or
    keyword-based, it's up to you. If you know the website and don't need a too general rule, you could just filter your
    links accordingly.


    LVL 1

    Author Comment

    Hello Andrew
    Thanks for your answer.How ever i want the code to  do the task at hand
    LVL 14

    Accepted Solution


      doc: IHTMLDocument2;
      links: IHTMLElementCollection;
      i: integer;
      doc := WebBrowser.Document as IHTMLDocument2;
      links := doc.Links;
      for i := 0 to links.length -1 do begin
             currlink := link.Item(i,'') as IHTMLElement;

    If you do not have MSHTML_TLB, create it by via Project->Import Type Library->Micorsoft HTML Object Library
    Deselect 'Generate Component Wrapper', and click 'Create Unit'.

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    Join & Write a Comment

    Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
    Creating an auto free TStringList The TStringList is a basic and frequently used object in Delphi. On many occasions, you may want to create a temporary list, process some items in the list and be done with the list. In such cases, you have to…
    Need more eyes on your posted question? Go ahead and follow the quick steps in this video to learn how to Request Attention to your question. *Log into your Experts Exchange account *Find the question you want to Request Attention for *Go to the e…
    This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor ( If you're looking for how to monitor bandwidth using netflow or packet s…

    734 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    25 Experts available now in Live!

    Get 1:1 Help Now