Solved

Submitting a file to a form on unix via cron

Posted on 2004-08-26
12
202 Views
Last Modified: 2009-07-29
All,

I have a windows web server and a unix db server, separated by a firewall.  I am looking to use a web form to copy 2 files hourly from the unix server to the windows server via an automated cron job.

The unix server has netscape and lynx, but lynx does not appear to work.  Perl is available, as is java.  I HAVE NO ROOT PRIVILEGES ON THIS BOX AND CANNOT GET ANY NEW SOFTWARE ADDED TO IT. (I really need wget, but I doubt it is available for AIX on RISC anyway)

What I need is a perl script, or another way to automate a form submission that includes files.  I cannot use FTP because 1) it is blocked at the firewall and 2) I need to process the files on the web server to append them to an Access db.

So,
Does anyone know how to simulate submitting files via a form with a single URL call?

my web page only outputs: <SCRIPT>window.close()</script> to the client.

My form is simple:
<form name=dbupload enctype="multipart/form-data" action="index.php" METHOD="POST">
Dump: <INPUT TYPE=file NAME=dbdump><br>
Tracks: <INPUT TYPE=file NAME=dbtracks><br>
<INPUT TYPE=submit value="Upload Files"><br>
</form>

HELP!!!!!!!!!!!!!!!!!!

0
Comment
Question by:oreomike
  • 5
  • 4
  • 2
  • +1
12 Comments
 
LVL 1

Author Comment

by:oreomike
ID: 11904190
I will GLADLY get someone another 500 points for answering this...
0
 
LVL 25

Expert Comment

by:Marcus Bointon
ID: 11904660
Can you use SSH? If so, you should be able to schedule an scp or rsync command to do the copy, both programs are available in most distributions. rsync would have the added advantage that it would only copy if files had been changed.

To persevere on the HTTP route from PHP, you can use CURL to submit form data via POST. First of all, get your file contents into a variable, then submit it as post data:

<?php
$data = urlencode(file_get_contents('myuploadfile1.txt'));
$URL="http://www.example.com/upload.php";
$ch = curl_init();   
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "data=$data");
curl_exec ($ch);    
curl_close ($ch);
?>

You'll need to tweak this to match the upload script on your windows box.
0
 
LVL 25

Expert Comment

by:Marcus Bointon
ID: 11904688
Revised version to match your form example:

<?php
$dbdump = urlencode(file_get_contents('dbdump'));
$dbtracks = urlencode(file_get_contents('dbtracks'));
$URL="http://www.example.com/index.php";
$ch = curl_init();   
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "dbdump=$dbdump&dbtracks=$dbtracks");
curl_exec ($ch);    
curl_close ($ch);
?>
0
 
LVL 1

Author Comment

by:oreomike
ID: 11904689
so I do not have PHP available either on the unix box.  Do you know if perl might have a similar mod as curl?
0
 
LVL 25

Assisted Solution

by:Marcus Bointon
Marcus Bointon earned 100 total points
ID: 11904793
CURL is a system level library and CLI application that happens to also have a PHP wrapper, so you can use it from pretty much anywhere, including perl and shell scripts. Try 'man curl' to see what it can do from a CLI (and thus a shell script). You'll find that the constants used in the PHP script are the same in all CURL implementations, so the above script may be of use anyway. Apart from that, I'm going to take a wild guess and suggest that the PHP TA is not the best place to ask about this!
0
 
LVL 48

Assisted Solution

by:hernst42
hernst42 earned 200 total points
ID: 11904798
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 1

Author Comment

by:oreomike
ID: 11904825
probably, but I didn't see a perl forum.  Thanks anyway.  I'll probably try a java cli instead.
0
 
LVL 1

Author Comment

by:oreomike
ID: 11904840
The unix box does not have the LWP:UserAgent package installed.
0
 
LVL 25

Expert Comment

by:Marcus Bointon
ID: 11904929
You'd be much better off going with a shell script than Java. Java is monstrously inefficient for small operations like this.
0
 

Accepted Solution

by:
IsonumericAl earned 200 total points
ID: 11933167
You could use Perl. You can install new Perl modules into your non-root account.

(Consider trying a web-testing program, they are able to submit to forms, etc., though they may not have the ability to submit a file: java's HTTPUnit).

INSTALLING

See http://www.hodgers.com/ali/programming/nonrootcpan.shtml to setup as "non-root install". And chose the CPAN servers that use HTTP.

Or, copy the current config, and edit all the paths that seem relevant.      
      # this tells you all the directories where .pm files are.
      # we want Config.pm, which was in /usr/lib/perl5/5.8.1/CPAN on my system
      perl -V
      # Copy...
      cd
      mkdir .cpan
      mkdir .cpan/CPAN
      # Fix the following path
      cp /usr/lib/perl5/5.8.1/CPAN/Config.pm .cpan/CPAN
      edit .cpan/CPAN

And install
      perl -MCPAN -e shell
      install G/GA/GAAS/libwww-perl-5.800.tar.gz
      quit


PERL SCRIPT

Assuming you have a reasonably modern Perl, you almost certainly have the libwww modules. Try this to check:
      perldoc HTTP::Request
      or
      perl -mHTTP::Request
If either works, you probably have what we need. You can read the "perldoc" to see the documentation (see also the perldoc for HTTP::Request::Commong, LWP::UserAgent).

I'd suggest doing a regular POST, with the file in the body: just like a browser would do. There is a limit to the amount of data you can put in the GET (query-string), officially something like 3k. Check the "method" in the HTML "form" tag.

Here's the steps (edit the url, filename, etc.):

      #!/usr/bin/perl
      # be a good programmer
      use strict;
      use warnings;

      # Need the module
      require LWP::UserAgent;

      # Setup the useragent (it does the HTTP protocol)
      my $ua = LWP::UserAgent->new;
      push @{ $ua->requests_redirectable }, 'POST'; # if you need redirect-on-post

      # do the request **EDIT THIS STUFF**
      # (see HTTP::Request::Common,
      my $response = $ua->post(
            # Your url here (from "action":
            'http://www.oslo.net/',
            # The form fields
            {
                  # A file (from disk) looks like this
                  theFile => [ 'nameOfFileOnDisk', 'nameToTellServer'],
                  # A "input" field (etc.) looks like this
                  address => '123 Anywhere St.',
            },
            ['form-data']      # a header for posting files (mime/multiplart)
            );

      # Check for success
      # Maybe you don't want to "die" here? You could do a "warn"
      # and the output would go to your email (from cron).
      die "Error while POSTING ", $response->request->uri,
            " -- ", $response->status_line, "\nAborting"
            unless $response->is_success
0
 

Expert Comment

by:IsonumericAl
ID: 11933277
More on installing perl modules as non-root:
http://www.bugzilla.org/docs/tip/html/nonroot.html#AEN931
0
 
LVL 1

Author Comment

by:oreomike
ID: 11945374
All of your suggestions were good, but given my limited ability to do anything on the box, including a limited FS size for the home directories, I was forced to go java.

I have included the code that I modified from http://java.sun.com to only upload a single file to a form with a single element.  Currently it only will work well with text files and may need a change to \n's to make them \r\n's.  Haven't had time to fully check it out.  Also, I am sure there is a better way to send contents of a file to an InputStream, but this gets my file there and I am happy.

Thanks for all of your quick and thorough help.

*********************************************************

import java.io.*;
import java.net.*;

public class uploadFile {

  public static void writeFile(DataOutputStream dstream,String filename,String formname) throws Exception {
      File f0 = new File(filename);
      try {
            FileInputStream fi = new FileInputStream(f0);
            dstream.writeBytes("Content-Disposition: form-data; name=\""+formname+"\"; filename=\""+f0.getName()+"\"\r\n");
            dstream.writeBytes("Content-Type: text/plain\r\n\r\n");
            int cnt = fi.read();
            while(cnt!=-1){
                  dstream.write(cnt);
                  cnt = fi.read();
            }
            fi.close();
            dstream.writeBytes("\r\n--****4353\r\n");
      } catch (FileNotFoundException e) {
            System.out.println("File not found: " + f0.getName());
      }
  }

  public static void main(String[] args) throws Exception {
      if (args.length != 5) {
          System.err.println(
            "Usage:\njava uploadFile http://this/url.php formInputBoxName /path/and/file.name");
          System.exit(1);
      }
      String strURL = args[0]
      URL url = new URL(strURL);
      URLConnection conn = url.openConnection();
      conn.setDoOutput(true);
      conn.setDoInput(true);
      conn.setUseCaches(false);
      conn.setRequestProperty("Connection","Keep-Alive");
      conn.setRequestProperty("HTTP_REFERER",strURL);
      conn.setRequestProperty("Content-Type","multipart/form-data; boundary=****4353");
      DataOutputStream dstream = new DataOutputStream(conn.getOutputStream());
      dstream.writeBytes("--****4353\r\n");

      writeFile(dstream,args[1],args[2]);

      dstream.writeBytes("\r\n--****4353--\r\n");
      dstream.flush();
      dstream.close();
      try{
            DataInputStream in = new DataInputStream( new BufferedInputStream(conn.getInputStream()));
            String sIn = in.readLine();
            boolean b = true;
            System.out.println(sIn);
            while(sIn.compareTo("window.close();")!=0){
                  sIn = in.readLine();
                  if(sIn!=null){
                        System.out.println(sIn);
                  }
            }

      } catch (Exception e) {
            e.printStackTrace();
      }
  }
}
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
Things That Drive Us Nuts Have you noticed the use of the reCaptcha feature at EE and other web sites?  It wants you to read and retype something that looks like this.Insanity!  It's not EE's fault - that's just the way reCaptcha works.  But it is …
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now