?
Solved

How to get the source of http document as fast as possible?

Posted on 2003-03-05
11
Medium Priority
?
228 Views
Last Modified: 2010-04-04
Hi!

 This site looks great to me, so I have registered allready! :-)

 My problem is:

 I want to get a html source of web document, but ONLY this. Using WebBrowser causes me to wait until the document will be diplayed. And I don't need it to be displayed.
 

  In Delphi5 I use this function:

function TForm1.page(url: string): string;
var
  Doc: IHTMLDocument2;
  Flags : OLEVariant;
begin
  Flags := 14;    // 2+4+8 = do not use cache and history
  WebBrowser1.Navigate(adres, Flags);
  repeat
    Application.ProcessMessages;
  until not WebBrowser1.Busy;
  Doc := WebBrowser1.Document AS IHTMLDocument2;
  if Doc <> nil then
    page := (Doc.Body AS IHtmlElement).innerHTML;
end;

 so I have a source of document using:
 source := page(http://www.some.site.on.the.net');

 I am sure there is some tricky way to obtain only a source, without displaying the whole www site.

 Maybe I can use somehow RequestDoc method?
 Anyway - I have no idea how to do this.

 Please, can you help me?

 Best regards
0
Comment
Question by:matejo
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
11 Comments
 

Accepted Solution

by:
Soko earned 152 total points
ID: 8078924
I think you could use wininet here is a link explaining how :
http://delphi.about.com/library/weekly/aa013001a.htm

You should use the URL of you web page and a filename like mywebpage.text you can then put it in a stringlist. Indeed you could also modify the code in order to get it directly in a stream for instance.

Wininet is underlying to Internet explorer so you might prefer using the very good freeware Internet Component suite by F Piette here
http://overbyte.delphicenter.com/frame_index.html

Hope this helps
0
 
LVL 2

Assisted Solution

by:Tvigep
Tvigep earned 148 total points
ID: 8081181
I use the Indy components (http://www.indyproject.org/), and with their TidHTTP component it's very easy and fast:

var source:string;
source:=TidHTTP.Get('http://www.some.site.on.the.net');

will get me just the html and nothing more. Since just the HTML is usually only a couple of Kb at most, this is extremely fast.

HTH,
Stef
0
 

Author Comment

by:matejo
ID: 8081236
Thanks Tvigep,

 actually I found a few hours ago one of the solution, that is:
 NMHTTP1.Body;

 It is VERY easy, just like yours. :-)
 But I am actually testing it.
 
 Your solution will also be tested. I'll chceck it.
 Thanky you all very much. The best solution (if it won't be mine) will be rewarded! :)

 Best regards
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:matejo
ID: 8081596
Thanks Tvigep,

 actually I found a few hours ago one of the solution, that is:
 NMHTTP1.Body;

 It is VERY easy, just like yours. :-)
 But I am actually testing it.
 
 Your solution will also be tested. I'll chceck it.
 Thanky you all very much. The best solution (if it won't be mine) will be rewarded! :)

 Best regards
0
 

Author Comment

by:matejo
ID: 8081748
All right then,

 my sloution works very well. Hmmm, it is not right that I got an answer from myself... ;-)

 The code looks like this:

 uses NMHttp

...
 NMHTTP1.Get('http://www.some.site.on.the.net);
 st := NMHTTP1.Body;
...

 Very easy, indeed.
====================================================
====================================================

 -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
 But now - ANOTHER QUESTION!!!
 -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*

 Is it possible (all the time not using WebBrowser) to get HTTP Body, if the site won't allow me to reach the HTML document without a Cookie?

 In other words: I receive a Cookie, which, using "normal" web browser (like IE or Netscape), allow me to surf on the site. But without the Cookie - nothing. Can I inform www.site that I have allready received a Cookie? How Can I do this? Is it possible at all?

 I rise the amount of point to 75 (all I have :) ) since it looks more complicated.

 Best regards!
0
 

Expert Comment

by:Soko
ID: 8081941
You can set the cookie password and everything you need like this (replace'' by the relevant string :

With NMHttp.HeaderInfo do
    Begin
      Cookie           := '';
      LocalMailAddress := '';
      LocalProgram     := '';
      Referer          := '';
      UserID           := '';
      Password         := '';
    End;
0
 

Expert Comment

by:Soko
ID: 8081989
I forgot to tell that to read a cookie send to you by the web site you have to read the header that will saved in the file specified by NMHttp.Header
0
 

Author Comment

by:matejo
ID: 8082037
Ha ha ha, I found it too in a help. :D :D

 Ok, if it'll work, you get your points! :)

 Best regards!
0
 

Expert Comment

by:CleanupPing
ID: 9316915
matejo:
This old question needs to be finalized -- accept an answer, split points, or get a refund.  For information on your options, please click here-> http:/help/closing.jsp#1 
EXPERTS:
Post your closing recommendations!  No comment means you don't care.
0
 
LVL 5

Expert Comment

by:snehanshu
ID: 10044666
Hi!
No comment has been added lately and this question is therefore classified abandoned.

If asker wishes to close the question, then refer to
http://www.experts-exchange.com/help/closing.jsp

Otherwise, I will leave a recommendation in the Cleanup topic area that this question is:

Split between Soko and Tvigep

Please leave any comments here within the next seven days. It is assumed that any participant not responding to this request is no longer interested in its final disposition.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!

...Snehanshu
EE Cleanup Volunteer
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Creating an auto free TStringList The TStringList is a basic and frequently used object in Delphi. On many occasions, you may want to create a temporary list, process some items in the list and be done with the list. In such cases, you have to…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
In this video you will find out how to export Office 365 mailboxes using the built in eDiscovery tool. Bear in mind that although this method might be useful in some cases, using PST files as Office 365 backup is troublesome in a long run (more on t…
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
Suggested Courses
Course of the Month10 days, 21 hours left to enroll

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question