How to catch information from web page using perl

Posted on 2005-04-15
Last Modified: 2010-03-05
Hi ,maybe I am asking a stupid question but

what I want to do is ,
I am having link to some web page that print on the screen some information
can I read this information to some file ?
Is it possible ?

Please explain ,how can I do this .
Question by:Vlad77
    LVL 18

    Expert Comment

    Here's some command lines you can use:

        wget http://your/link/here
        # file is stored in 'here'

        perl -MLWP::Simple -e 'getprint("http://your/link/here")' > your/file/here

        GET http://your/link/here > your/file/here

    Author Comment

    This is a server that require username and password ,

    How can I insert user and pass in perl using this wget or get commands ?
    LVL 20

    Expert Comment

    That will probably depend on how the username and password are authenticated to the website. One possibility is that it is using what's called BASIC authentication, in which case you can supply the username and password in the URL:


    For other kinds of authentication (and we may be able to help you decide what kind is in use) you would most likely need to write a somewhat more complicated LWP script.

    Accepted Solution

    $req = GET 'http://XXXXXX/cgi-bin/';

    $req->authorization_basic('login', 'pass'); $response = $ua->request( $req );

    Expert Comment

    None of these work on my server.  Am I missing perl addon modules? I have a link just like which works fine in a browser. I just want perl to read it.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    What Security Threats Are You Missing?

    Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

    Suggested Solutions

    Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
    Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
    Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
    This video discusses moving either the default database or any database to a new volume.

    779 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now