Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 206
  • Last Modified:

Better/faster way to access the internet for scraping?

I am presently using WebRequest and WebResponse to access a particular web site, which is very slow compared to accessing by browser.  Is there a better/faster way to access a web page for scraping?
0
rmmarsh
Asked:
rmmarsh
1 Solution
 
bruintjeCommented:
Hello rmmarsh,

if it is faster i do not know that also depends on how you parse the contents, but you could look at the webclient class in .NET
source: http://www.codersource.net/csharp_screen_scraping.html

that sample even gives a small hint about extracting parts with regular expressions in this case to extract all links on the page

hope this helps a bit
bruintje
0

Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now