Solved

Scraping Info From a website and saveing it to my computer.'Spyder' a site.

Posted on 2006-11-14
7
377 Views
Last Modified: 2010-04-04
Hello I am working on a project were i need to go to a webpage and save all the job postings. Example belowl
http://www.reed.co.uk/JobSearchResults.aspx?SearchType=Simple&SearchRoute=20&JobSector=4
. I need to scrape all the job postings from that page , The project goal is to spyder to whole site and scrap all the job postings and save to my computer.
Thanks
Grant
0
Comment
Question by:Grant Fullen
  • 3
  • 3
7 Comments
 

Author Comment

by:Grant Fullen
ID: 17946519
Hey just checking in. Any thoughts anyone.
Thanks
Grant
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17949114
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17949137
Or check out Felix VColibri's Web Spider code:
http://www.felix-colibri.com/papers/web/web_spider/web_spider.html

Beware that the code is not very readable, but that's my opinion.
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 

Author Comment

by:Grant Fullen
ID: 17951106
Those are good links. I know how to save the html but I really need to know how to spider all the categories automaticly.
Thanks
Grant
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17955922
The code in both posts show you how. All it takes is a little imagination.
0
 
LVL 9

Accepted Solution

by:
bernani earned 500 total points
ID: 17961888
Hi,

Maybe you can use the TEmbeddedWB
Freeware Component
For Delphi 4, 5, 6, 7, 2005, 2006
Documentation and updated versions:  http://www.bsalsa.com  

In the huge demo of the capabilities of the compo, you have those procedures:

procedure TfrmMain.ViewPageLinksAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageLinksToStrings(RichEditWB1.Lines);
end;

procedure TfrmMain.ViewPageImagesAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageImagesToStrings(RichEditWB1.Lines);
end;

procedure TfrmMain.ViewPageFieldsAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageFieldsToStrings(RichEditWB1.Lines);
end;


I've tested them to the page you supplied and the result is as follow after changing the No. of results per page  to 100 isntead of 10   :

(result of the procedure procedure TfrmMain.ViewPageLinksAsAList1Click(Sender: TObject);

http://www.reed.co.uk/
http://www.reed.co.uk/default.aspx
http://www.reed.co.uk/tempzone/default.aspx
http://www.reed.co.uk/freecruitment.aspx
http://www.reed.co.uk/employers/default.aspx
http://www.reed.co.uk/default.aspx
https://www.reed.co.uk/login.aspx?referrer=strapline
https://www.reed.co.uk/register
http://www.reed.co.uk/sectorsMenu.aspx
http://www.reed.co.uk/localjobs.aspx
http://www.reed.co.uk/toolsMenu.aspx
http://www.reed.co.uk/contactMenu.aspx
http://www.reed.co.uk/help/default.aspx
javascript:void(openWindow('/keywordpopup.aspx','Keywordpopup','680','490'))
http://www.reed.co.uk/jobrefsearch.aspx
http://www.reed.co.uk/AdvancedJobSearch.aspx
http://www.reed.co.uk/confidence.aspx
http://www.reed.co.uk/JobSearchResults.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831
http://www.reed.co.uk/SaveJobSearch.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1
http://www.reed.co.uk/AdvancedJobSearch.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblResultsTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblReedTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblTodaysTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblNotAppliedTab','')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9446151&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468124&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10112189&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468013&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10385101&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371502&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10356610&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10363188&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10301167&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10274315&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9914148&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467800&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467781&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9369457&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467752&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10454964&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10396757&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10378674&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10319332&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10303598&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10381446&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10381344&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10387278&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10387212&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10423406&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10423352&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10189935&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10165091&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10163509&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10162230&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10120788&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10102255&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10096389&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10092547&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10073408&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10066053&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10060337&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10026173&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9893823&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9886102&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9664664&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9420898&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9114661&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10327597&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10314541&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10119524&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9884336&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467508&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467505&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467495&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467207&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467192&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9353459&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523647&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523589&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523433&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9661036&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9707634&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9724301&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467286&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10258549&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9818494&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9834373&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10065127&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467112&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10196790&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10415893&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10460586&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467018&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10466988&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10433506&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10294500&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10312474&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10308667&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10294488&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9741683&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10065160&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10396956&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10326352&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10204020&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9684775&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9684581&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10322359&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9910647&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9301385&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10466768&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9909041&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10465794&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9909016&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10354902&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10370111&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10369615&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10426572&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10426526&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10439475&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10439400&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
http://www.reed.co.uk/siteMap.aspx
http://www.reed.co.uk/diversity.aspx
javascript:void(window.external.AddFavorite('http://www.reed.co.uk/','reed.co.uk'))
http://www.reed.co.uk/siteInfo.aspx
http://www.reed.co.uk/terms.aspx
http://www.reed.co.uk/privacy.aspx
http://www.reed.co.uk/workForReed/
http://www.reed.co.uk/ExternalBanner.aspx?pro=www.reed.co.uk&target=http://www.reed.co.uk/training/
http://www.reed.co.uk/ExternalBanner.aspx?pro=www.reed.co.uk&target=http://www.reedconsulting.co.uk/

the 100 results of your search for this page(Page 1) are in the lines above those one:

http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9446151&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468124&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10112189&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468013&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10385101&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371502&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False

and so on ...

Why:
if you click on the first item:
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False

your have the content of the field displayed.

Accounts Assistant Next »
 
Agency: Kelvin Alexander Recruitment
Salary Negotiable Manchester, North West
Type: Permanent Ref: 9829831
Date: 16 Nov Applications: 22
--------------------------------------------------------------------------------
Main Duties:-

1. Show Settlement – producing statement of sales for shows and issuing cheques against statements and invoices received from promoters.

.....
 

You thus have only to download the content of 100 links above (you can fill a ListBox or a Stringlist or file or what you want with each set of 100 adresses) and repeat the same for each page.


You can do it autmatically.

If you right click on page  2 -----> 1|2| , you see that the adress is the following:
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')

Paste this adress:
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
in the adress bar of your browser and you got the second page display (or in your project load it programatically).

Repeat the same steps for page 2 as you did for page 1 to gather all the remaining links.

At the end, you have got a list with X urls to download to a file, in a db, as records, ....

Use the procedure Eddie gave above in one's of his previous post (http://www.experts-exchange.com/Programming/Programming_Languages/Delphi/Q_21555641.html ---> UrlDownloadToFile) to save all the urls you have retrieved in a file, db, record, ... (the name could be the JobID (JobID=10370111) to parse from the url.

Hope those explanations can help you to grab the docs you need and I believe a demo is not really necessary .





0
 

Author Comment

by:Grant Fullen
ID: 17969715
Thanks alot for takeing the time to help me find a solution.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
As a trusted technology advisor to your customers you are likely getting the daily question of, ‘should I put this in the cloud?’ As customer demands for cloud services increases, companies will see a shift from traditional buying patterns to new…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now