Solved

Scraping Info From a website and saveing it to my computer.'Spyder' a site.

Posted on 2006-11-14
7
379 Views
Last Modified: 2010-04-04
Hello I am working on a project were i need to go to a webpage and save all the job postings. Example belowl
http://www.reed.co.uk/JobSearchResults.aspx?SearchType=Simple&SearchRoute=20&JobSector=4
. I need to scrape all the job postings from that page , The project goal is to spyder to whole site and scrap all the job postings and save to my computer.
Thanks
Grant
0
Comment
Question by:Grant Fullen
  • 3
  • 3
7 Comments
 

Author Comment

by:Grant Fullen
ID: 17946519
Hey just checking in. Any thoughts anyone.
Thanks
Grant
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17949114
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17949137
Or check out Felix VColibri's Web Spider code:
http://www.felix-colibri.com/papers/web/web_spider/web_spider.html

Beware that the code is not very readable, but that's my opinion.
0
Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

 

Author Comment

by:Grant Fullen
ID: 17951106
Those are good links. I know how to save the html but I really need to know how to spider all the categories automaticly.
Thanks
Grant
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 17955922
The code in both posts show you how. All it takes is a little imagination.
0
 
LVL 9

Accepted Solution

by:
bernani earned 500 total points
ID: 17961888
Hi,

Maybe you can use the TEmbeddedWB
Freeware Component
For Delphi 4, 5, 6, 7, 2005, 2006
Documentation and updated versions:  http://www.bsalsa.com  

In the huge demo of the capabilities of the compo, you have those procedures:

procedure TfrmMain.ViewPageLinksAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageLinksToStrings(RichEditWB1.Lines);
end;

procedure TfrmMain.ViewPageImagesAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageImagesToStrings(RichEditWB1.Lines);
end;

procedure TfrmMain.ViewPageFieldsAsAList1Click(Sender: TObject);
begin
   UpdateEditor;
   EmbeddedWB1.ViewPageFieldsToStrings(RichEditWB1.Lines);
end;


I've tested them to the page you supplied and the result is as follow after changing the No. of results per page  to 100 isntead of 10   :

(result of the procedure procedure TfrmMain.ViewPageLinksAsAList1Click(Sender: TObject);

http://www.reed.co.uk/
http://www.reed.co.uk/default.aspx
http://www.reed.co.uk/tempzone/default.aspx
http://www.reed.co.uk/freecruitment.aspx
http://www.reed.co.uk/employers/default.aspx
http://www.reed.co.uk/default.aspx
https://www.reed.co.uk/login.aspx?referrer=strapline
https://www.reed.co.uk/register
http://www.reed.co.uk/sectorsMenu.aspx
http://www.reed.co.uk/localjobs.aspx
http://www.reed.co.uk/toolsMenu.aspx
http://www.reed.co.uk/contactMenu.aspx
http://www.reed.co.uk/help/default.aspx
javascript:void(openWindow('/keywordpopup.aspx','Keywordpopup','680','490'))
http://www.reed.co.uk/jobrefsearch.aspx
http://www.reed.co.uk/AdvancedJobSearch.aspx
http://www.reed.co.uk/confidence.aspx
http://www.reed.co.uk/JobSearchResults.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831
http://www.reed.co.uk/SaveJobSearch.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1
http://www.reed.co.uk/AdvancedJobSearch.aspx?JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblResultsTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblReedTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblTodaysTab','')
javascript:__doPostBack('ccJobSearchResults$_ctl0$regionMainPage$CompositeResults$ResultsTabControl$lblNotAppliedTab','')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9446151&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468124&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10112189&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468013&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10385101&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371502&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10356610&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10363188&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10301167&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10274315&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9914148&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467800&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467781&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9369457&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467752&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10454964&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10396757&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10378674&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10319332&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10303598&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10381446&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10381344&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10387278&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10387212&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10423406&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10423352&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10189935&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10165091&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10163509&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10162230&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10120788&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10102255&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10096389&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10092547&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10073408&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10066053&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10060337&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10026173&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9893823&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9886102&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9664664&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9420898&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9114661&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10327597&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10314541&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10119524&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9884336&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467508&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467505&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467495&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467207&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467192&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9353459&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523647&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523589&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9523433&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9661036&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9707634&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9724301&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467286&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10258549&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9818494&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9834373&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10065127&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467112&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10196790&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10415893&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10460586&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10467018&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10466988&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10433506&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10294500&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10312474&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10308667&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10294488&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9741683&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10065160&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10396956&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10326352&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10204020&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9684775&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9684581&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10322359&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9910647&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9301385&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10466768&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9909041&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10465794&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9909016&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10354902&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10370111&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10369615&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10426572&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10426526&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10439475&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10439400&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
http://www.reed.co.uk/siteMap.aspx
http://www.reed.co.uk/diversity.aspx
javascript:void(window.external.AddFavorite('http://www.reed.co.uk/','reed.co.uk'))
http://www.reed.co.uk/siteInfo.aspx
http://www.reed.co.uk/terms.aspx
http://www.reed.co.uk/privacy.aspx
http://www.reed.co.uk/workForReed/
http://www.reed.co.uk/ExternalBanner.aspx?pro=www.reed.co.uk&target=http://www.reed.co.uk/training/
http://www.reed.co.uk/ExternalBanner.aspx?pro=www.reed.co.uk&target=http://www.reedconsulting.co.uk/

the 100 results of your search for this page(Page 1) are in the lines above those one:

http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9827531&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10027005&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=9446151&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468124&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10112189&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10468013&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10385101&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371502&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False
http://www.reed.co.uk/JobDetails.aspx?JobID=10371411&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False

and so on ...

Why:
if you click on the first item:
http://www.reed.co.uk/JobDetails.aspx?JobID=9829831&JobSearchGUID=535274DE-0385-40BE-9863-3D1B33C926BC&Page=1&From=13&ReedSearch=False

your have the content of the field displayed.

Accounts Assistant Next »
 
Agency: Kelvin Alexander Recruitment
Salary Negotiable Manchester, North West
Type: Permanent Ref: 9829831
Date: 16 Nov Applications: 22
--------------------------------------------------------------------------------
Main Duties:-

1. Show Settlement – producing statement of sales for shows and issuing cheques against statements and invoices received from promoters.

.....
 

You thus have only to download the content of 100 links above (you can fill a ListBox or a Stringlist or file or what you want with each set of 100 adresses) and repeat the same for each page.


You can do it autmatically.

If you right click on page  2 -----> 1|2| , you see that the adress is the following:
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')

Paste this adress:
javascript:__doPostBack('ccJobSearchResults:_ctl0:regionMainPage:CompositeResults:ResultsTableControl','2')
in the adress bar of your browser and you got the second page display (or in your project load it programatically).

Repeat the same steps for page 2 as you did for page 1 to gather all the remaining links.

At the end, you have got a list with X urls to download to a file, in a db, as records, ....

Use the procedure Eddie gave above in one's of his previous post (http://www.experts-exchange.com/Programming/Programming_Languages/Delphi/Q_21555641.html ---> UrlDownloadToFile) to save all the urls you have retrieved in a file, db, record, ... (the name could be the JobID (JobID=10370111) to parse from the url.

Hope those explanations can help you to grab the docs you need and I believe a demo is not really necessary .





0
 

Author Comment

by:Grant Fullen
ID: 17969715
Thanks alot for takeing the time to help me find a solution.
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Mydac connection data base issue 3 157
Delphi Firemonkey Need Sample for Online Shopping Example. 2 142
How to debug For loops? 3 49
DBGrid or StringGrid ? 6 88
Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
Email security requires an ever evolving service that stays up to date with counter-evolving threats. The Email Laundry perform Research and Development to ensure their email security service evolves faster than cyber criminals. We apply our Threat…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question