• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2855
  • Last Modified:

Difference between WGET and CURL command and find equivalent solution

What is the difference between these two commands?
1) wget
2) curl

I have wget command in this format: wget --spider -t2 -T3 -F -i ${file}.html 2> output.txt
Basically, this wget will parse through all the links in a html page and return the results into the output.txt file.

I am trying to do same thing using CURL. Does anybody has solution for this. The reason for me to switch to CURL or something else which is similar to WGET is that WGET is getting stuck on one particular file which has multiple redirect links (about 20) which are all same and it's just gets stuck..

Hopefully CURL will provide better solution if there is a way to do it.
Thanks.
0
desiaashik
Asked:
desiaashik
1 Solution
 
slyongCommented:
Hi desiaashik,

cURL doesn't do things like wget --spider as decribed in the FAQ (http://curl.mirrors.cyberservers.net/docs/faq.html#3.15).  There are a lot of crawlers/spiders on Linux that you can use, an example would be http://www.w3.org/Robot/
0

Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now