Solved

Difference between WGET and CURL command and find equivalent solution

Posted on 2007-04-11
1
2,586 Views
Last Modified: 2012-08-13
What is the difference between these two commands?
1) wget
2) curl

I have wget command in this format: wget --spider -t2 -T3 -F -i ${file}.html 2> output.txt
Basically, this wget will parse through all the links in a html page and return the results into the output.txt file.

I am trying to do same thing using CURL. Does anybody has solution for this. The reason for me to switch to CURL or something else which is similar to WGET is that WGET is getting stuck on one particular file which has multiple redirect links (about 20) which are all same and it's just gets stuck..

Hopefully CURL will provide better solution if there is a way to do it.
Thanks.
0
Comment
Question by:desiaashik
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
 
LVL 24

Accepted Solution

by:
slyong earned 125 total points
ID: 18894857
Hi desiaashik,

cURL doesn't do things like wget --spider as decribed in the FAQ (http://curl.mirrors.cyberservers.net/docs/faq.html#3.15).  There are a lot of crawlers/spiders on Linux that you can use, an example would be http://www.w3.org/Robot/
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

A short article about problems I had with the new location API and permissions in Marshmallow
Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question