Solved

Is escaping required?

Posted on 2008-10-13
20
213 Views
Last Modified: 2010-03-05
Dear Experts,

My script does not always seem to work, and I can only assume it is a problem with not escaping.

The only place where I might need to escape the characters is on this line:

Subject: New email titled ($Row->{Heading})

Please help
0
Comment
Question by:narmi2
  • 10
  • 8
  • 2
20 Comments
 
LVL 1

Author Comment

by:narmi2
ID: 22701930
Would it help if I posted the full script?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22703308
We don't necessarily need to see your whole script, but you should at least post the relevant portions.

You also will need to tell us what errors and/or warnings your script is generating.

What output is it generating and how does that differ from the output you expect.
0
 
LVL 1

Author Comment

by:narmi2
ID: 22703574
Basically, I have an sql query run before a loop is run to send out emails.

On a specific day, lets say 2008-10-10 the sql query when run from a webpage gives me 3 rows of results.  This can be varified by using the script below:

#!/usr/bin/perl -w use DBI;

my $dsn = 'DBI:mysql:database:server';
my $user = 'user';
my $pass = 'pass';

my $dbh = DBI->connect($dsn, $user, $pass) or die "Can not connect to the DB: $DBI::errstr\n";

$SQL= "select ne.*, nd.Heading, nd.ArticleDate
from Table1 ne
inner join Table2 nc on ne.CategoryID = nc.CategoryID
inner join Table3 nd on nc.DirectNewsID = nd.DirectNewsID and nd.ArticleDate = '2008-10-10'";

$Select = $dbh->prepare($SQL);
$Select->execute();

print "Content-type:text/html\n\n";

while($Row=$Select->fetchrow_hashref)
{
      print "Heading: $Row->{Heading}<br />Email: $Row->{UserEmail}<br />Date: $Row->{ArticleDate}<br />";
}

exit 0;

This will return 3 rows on the screen

However, when I add script to this to send out emails per loop, it does not always send out the correct amount of emails.  3 in this case.  For example, using the query above with the date 2008-10-10 it only sends out 1 email.

Here is the script:

#!/usr/bin/perl -w use DBI;

my $dsn = 'DBI:mysql:database:server';
my $user = 'user';
my $pass = 'pass';

my $dbh = DBI->connect($dsn, $user, $pass) or die "Can not connect to the DB: $DBI::errstr\n";

$SQL= "select ne.*, nd.Heading, nd.ArticleDate
from Table1 ne
inner join table2 nc on ne.CategoryID = nc.CategoryID
inner join Table3 nd on nc.DirectNewsID = nd.DirectNewsID and nd.ArticleDate = curdate()";

$Select = $dbh->prepare($SQL);
$Select->execute();

while($Row=$Select->fetchrow_hashref)
{
$Mailer = '/usr/sbin/sendmail -t';
open MAIL,"|$Mailer";
print MAIL <<THE_EMAIL;
Reply-to: narmi1\@narmi.com,narmi2\@narmi.com
From: webserver\@narmi.com
To: $Row->{UserEmail}
Subject: New web article ($Row->{Heading})Subject here
THE_EMAIL
close MAIL;
}

print "Content-type:text/html\n\n";
exit 0;

Any idea why this is happening?
0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 22703717
A "pause" might help if the mail server doesn't like 3 emails coming from the same person with in .01 seconds.  What happens with you just run the sendmail command 3 times in a script, but NOT in a loop?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22704005
First, some recommended adjustments.

1) Don't put the use statements on the shebang line.

2) Instead of using -w switch, use the warnings pragma.

3) Use the strict prqgma and declare ALL of your vars.

4) Use proper code indentation within the blocks.

5) ALWAYS check the return code of the open call to make sure it was successful, and take action if it wasn't.

6) Opening a pipe to sendmail is ok, but my preference is to use MIME::Lite.

6) You're currently outputting a blank page, instead you should output at least some status info.

#!/usr/bin/perl
 

use warnings;

use strict;

use DBI;

use CGI;

use MIME::Lite;
 

my $cgi = CGI->new;

my $dsn = 'DBI:mysql:database:server';

my $user = 'user';

my $pass = 'pass';
 

$|++;

print $cgi->header,

      $cgi->start_html;
 

my $dbh = DBI->connect($dsn, $user, $pass)

               or die "Can not connect to the DB: $DBI::errstr\n";
 

my $SQL= "select ne.*, nd.Heading, nd.ArticleDate

from Table1 ne

inner join table2 nc on ne.CategoryID = nc.CategoryID

inner join Table3 nd on nc.DirectNewsID = nd.DirectNewsID and nd.ArticleDate = curdate()";
 

my $Select = $dbh->prepare($SQL);

$Select->execute();
 

while( my $Row = $Select->fetchrow_hashref ) {

    my $msg = MIME::Lite->new(

			'Reply-To' => 'narmi1@narmi.com, narmi2@narmi.com',

			From       => 'webserver@narmi.com',

			To         => $Row->{UserEmail},

			Subject    => "New web article ($Row->{Heading})Subject here",

			Data       => 'Body of email message'

    );

    

    print $cgi->h1("Sending email to $Row->{UserEmail}");

    $msg->send or print $cgi->h2("Failed to send email to $Row->{UserEmail}");

}

$dbh->disconnect;

$cgi->end_html;

Open in new window

0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22704051
I forgot 1 thing.  The output of die statements won't be sent to the browser, unless you specifically redirect them, which can be done with the CGI::Carp module.

use CGI::Carp qw(fatalsToBrowser);
0
 
LVL 1

Author Comment

by:narmi2
ID: 22712762
Hello,

I have had a chance to try this new script but it gave me the following error:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, user@server and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.
Apache/2.0.46 (Red Hat) Server at www.narmi.com Port 80

I used dos2unix to convert it to the currect file type for the linux server and also I gave it the rights 755.  Any idea why it is not working?

Question: Why does the subject section use double quotes and why do the other sections i.e. Reply-To, From, use single quotes?
And do I not need to escape the @ signs in the email addresses?
0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 22743391
Initially, were you saying it does work sometimes and doesn't others?  Did you try it without a loop?  

I believe the single vs double is to hold off variable substitution.  I think if you use single, they no variables will be substituted, and double does.  Or it could be vice-versa.  
0
 
LVL 1

Author Comment

by:narmi2
ID: 22766909
How do I not use a loop as I need to loop through the records from the database inorder to get the email addresses.
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22767260
If you received the "Internal Server Error" when executing the script I posted, then most likely you don't have the MIME::Lite module installed on the system.  Have you checked the web server error log?

You can verify if the module is installed by running this Perl command from the command prompt.

perl -MMIME::Lite -e 1

If you receive any output from that command, it means that the module needs to be installed.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 1

Author Comment

by:narmi2
ID: 22774864
Thanks.

I've had that module installed and no longer get the error I described above.  However, when I try to run that cgi file now via a browser, it just displays a blank screen which is expected, but it is not sending out any emails.

Any ideas on how to debug?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22775580
Are you using the script I posted?  If so, a blank page would not be expected unless the select statement failed to return any records.  Otherwise it would have printed out a status line for each $Row->{UserEmail} address.

Please post the exact script that you are using so we can know how and where to add some additional debugging.
0
 
LVL 1

Author Comment

by:narmi2
ID: 22776292
You're right, I made a tiny mistake which I have now corrected, and on the webpage, it displays a list of all the emails addresses where the emails should be sent.  It shows all the correct emails.

However, it did not send the emails out.

The script is as follows:
#!/usr/bin/perl

 

use warnings;

use strict;

use DBI;

use CGI;

use MIME::Lite;

 

my $cgi = CGI->new;

my $dsn = 'DBI:mysql:database:server';

my $user = 'user';

my $pass = 'pass';

 

$|++;

print $cgi->header,

      $cgi->start_html;

 

my $dbh = DBI->connect($dsn, $user, $pass)

               or die "Can not connect to the DB: $DBI::errstr\n";

 

my $SQL= "select ne.*, nd.Heading, nd.ArticleDate

from Table1 ne

inner join Table2 nc on ne.CategoryID = nc.CategoryID

inner join Table3 nd on nc.DirectNewsID = nd.DirectNewsID and nd.ArticleDate = '2008-10-21'";

 

my $Select = $dbh->prepare($SQL);

$Select->execute();

 

while( my $Row = $Select->fetchrow_hashref ) {

    my $msg = MIME::Lite->new(

			'Reply-To' => 'narmi1@narmi.com, narmi2@narmi.com',

			From       => 'webserver@narmi.com',

			To         => $Row->{UserEmail},

			Subject    => "New web article ($Row->{Heading})",

			Data       => 'body goes here'

    );

    

    print $cgi->h1("Sending email to $Row->{UserEmail}");

    $msg->send or print $cgi->h2("Failed to send email to $Row->{UserEmail}");

}

$dbh->disconnect;

$cgi->end_html;

Open in new window

0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22776771
Try this version, I added a little debugging and commented out the Reply-To line.

Then post back with the results of its output.
#!/usr/bin/perl

 

use warnings;

use strict;

use DBI;

use CGI;

use MIME::Lite;

use Data::Dumper;

 

my $cgi = CGI->new;

my $dsn = 'DBI:mysql:database:server';

my $user = 'user';

my $pass = 'pass';

 

$|++;

print $cgi->header,

      $cgi->start_html;

 

my $dbh = DBI->connect($dsn, $user, $pass)

               or die "Can not connect to the DB: $DBI::errstr\n";

 

my $SQL= "select ne.*, nd.Heading, nd.ArticleDate

from Table1 ne

inner join Table2 nc on ne.CategoryID = nc.CategoryID

inner join Table3 nd on nc.DirectNewsID = nd.DirectNewsID and nd.ArticleDate = '2008-10-21'";

 

my $Select = $dbh->prepare($SQL);

$Select->execute();

 

while( my $Row = $Select->fetchrow_hashref ) {

    my $msg = MIME::Lite->new(

                      #  'Reply-To' => 'narmi1@narmi.com, narmi2@narmi.com',

                        From       => 'webserver@narmi.com',

                        To         => $Row->{UserEmail},

                        Subject    => "New web article ($Row->{Heading})",

                        Data       => 'body goes here'

    );

    

    print $cgi->h1("Sending email to $Row->{UserEmail}");

    $msg->send or print $cgi->h2("Failed to send email to $Row->{UserEmail}");

    

    print '<pre>';

    print Dumper $Row;

    print '</pre><br><br>';
 

}

$dbh->disconnect;

$cgi->end_html;

Open in new window

0
 
LVL 1

Author Comment

by:narmi2
ID: 22776895
Before I try that.  I just received all the emails....
1 hour later........

For for that hour, the server was kind of slow...

Related problems?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 22777021
No, this script should not cause that kind of delay.

If you're the admin of the mail server, as a start you might want to look at the mail queue and running process to see what might be causing the delay.
0
 
LVL 1

Author Comment

by:narmi2
ID: 22777073
Unfortunately, I am the admin'ish... I was given the server with no experience of Linux and told "here you go, start developing"... So, with that in mind, any idea how to check the mail queue?

I've try your latest code.
0
 
LVL 1

Author Comment

by:narmi2
ID: 22777214
OK, the output is as follows:

Sending email to email1@mail.com,email2@mail.com

$VAR1 = {
          'ID' => '22',
          'ArticleDate' => '2008-10-21 00:00:00',
          'Heading' => 'heading 1',
          'UserEmail' => 'email1@mail.com,email2@mail.com',
          'CategoryID' => '123'
        };



Sending email to email3@mail.com,email4@mail.com,email5@mail.com
$VAR1 = {
          'ID' => '12',
          'ArticleDate' => '2008-10-21 00:00:00',
          'Heading' => 'heading 2',
          'UserEmail' => 'email3@mail.com,email4@mail.com,email5@mail.com',
          'CategoryID' => '321'
        };

And it seems to send the emails instantly.

As it seems to be working, do I need to make any changes to the script for a live production environment?

Thanks for all the help.
0
 
LVL 28

Accepted Solution

by:
FishMonger earned 500 total points
ID: 22777682
The script is "ok" for production use, but there are some improvements that I'd recommend.

The error handling on the connect statement should be reworked.  As is, it "rudely" exits without giving any info to the user; the error message goes to stderr which is not being sent to the browser.  You could write a custom die error handler or, instead of using a die statement, you could execute a subroutine that displays a proper error message then exits.

You should also check that the prepare and execute statements are successful and take proper action if one or the other fails.

You should add checks to make sure that it actually returned at least 1 record and give an appropriate status message.
0
 
LVL 1

Author Comment

by:narmi2
ID: 22777895
That is perfect, Thanks!

The true test to see if this is working will happen in about 15 minutes when the cronjob runs this automatically.
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now