Solved

Difference between WGET and CURL command and find equivalent solution

Posted on 2007-04-11
1
2,502 Views
Last Modified: 2012-08-13
What is the difference between these two commands?
1) wget
2) curl

I have wget command in this format: wget --spider -t2 -T3 -F -i ${file}.html 2> output.txt
Basically, this wget will parse through all the links in a html page and return the results into the output.txt file.

I am trying to do same thing using CURL. Does anybody has solution for this. The reason for me to switch to CURL or something else which is similar to WGET is that WGET is getting stuck on one particular file which has multiple redirect links (about 20) which are all same and it's just gets stuck..

Hopefully CURL will provide better solution if there is a way to do it.
Thanks.
0
Comment
Question by:desiaashik
1 Comment
 
LVL 24

Accepted Solution

by:
slyong earned 125 total points
ID: 18894857
Hi desiaashik,

cURL doesn't do things like wget --spider as decribed in the FAQ (http://curl.mirrors.cyberservers.net/docs/faq.html#3.15).  There are a lot of crawlers/spiders on Linux that you can use, an example would be http://www.w3.org/Robot/
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
Computer science students often experience many of the same frustrations when going through their engineering courses. This article presents seven tips I found useful when completing a bachelors and masters degree in computing which I believe may he…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…

832 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question