Avatar of faithless1
faithless1 asked on

Content Extraction

Hi,

I'm looking for a way extract content from a file containing a list of URLs on each line and  output all content to a file.

Thank you.
Perl

Avatar of undefined
Last Comment
Fero45

8/22/2022 - Mon
Minh Võ Công

Minh Võ Công

TEST = "test.txt";
open(TEST) or die("Could not open log file.");
foreach $line (<TEST>) {
    chomp($line);              # remove the newline from $line.
    # do line-by-line processing.
}
Tintin

What format is the file with URL's in and what format do you want the output to be?
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
ASKER
faithless1

Both files are .txt format

Thank you
Tintin

It's still not clear what you are wanting to do.

OK, you have a file with a list of URL's.  Are they listed one per line, eg:

http://example.com
http://example.com/page

Then, what do you want written to the new file?
ASKER
faithless1

Hi,

I want to extract all content from each URL and pipe the results to a .txt file. Only words and numbers and exclude images.

Thank you
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
ASKER
faithless1

Minhvc,

I'm getting this error:
Can't modify constant item in scalar assignment at content.pl line 9, near ""test.txt";"
Global symbol "$line" requires explicit package name at content.pl line 11.
Global symbol "$line" requires explicit package name at content.pl line 12.
Bareword "TEST" not allowed while "strict subs" in use at content.pl line 9.
Execution of content.pl aborted due to compilation errors.

When running:

#!/usr/local/bin/perl

use strict;
use warnings;




TEST = "test.txt";
open(TEST) or die("Could not open log file.");
foreach $line (<TEST>) {
    chomp($line);              # remove the newline from $line.
    # do line-by-line processing.
}
ASKER CERTIFIED SOLUTION
Fero45

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
See how we're fighting big data
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question