Solved

Difference between WGET and CURL command and find equivalent solution

Posted on 2007-04-11
1
2,554 Views
Last Modified: 2012-08-13
What is the difference between these two commands?
1) wget
2) curl

I have wget command in this format: wget --spider -t2 -T3 -F -i ${file}.html 2> output.txt
Basically, this wget will parse through all the links in a html page and return the results into the output.txt file.

I am trying to do same thing using CURL. Does anybody has solution for this. The reason for me to switch to CURL or something else which is similar to WGET is that WGET is getting stuck on one particular file which has multiple redirect links (about 20) which are all same and it's just gets stuck..

Hopefully CURL will provide better solution if there is a way to do it.
Thanks.
0
Comment
Question by:desiaashik
1 Comment
 
LVL 24

Accepted Solution

by:
slyong earned 125 total points
ID: 18894857
Hi desiaashik,

cURL doesn't do things like wget --spider as decribed in the FAQ (http://curl.mirrors.cyberservers.net/docs/faq.html#3.15).  There are a lot of crawlers/spiders on Linux that you can use, an example would be http://www.w3.org/Robot/
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question