Solved

Powerscript to C#

Posted on 2012-12-27
4
294 Views
Last Modified: 2013-01-02
The attached file is a sample script that will login to a site then using
and index file , which is simple delimited text,  it  retrieves
pdf's.
Would greatly appreciate it if someone knows the C# or vb.net equivalent
code for these parts:   post the  username and password, get the  cookie, and then
request the PDF's , using a URL  that's in the index.
Looks to me that the  request for this has to include the  cookie in the
header, otherwise it just gets redirected to the login page.

Reason I don't want to just use the script is that I need to be able to load the index into a
DB and filter out only the ones we want. Sure, I could rewrite the index file, but
a  .net version would be very handy to have.

So, this part
"the C# or vb.net equivalent
code that will post the login creds, get the  cookie, and then
request the PDF's , using a URL  "


Thanks
0
Comment
Question by:awalkinthepark
  • 2
  • 2
4 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 38724846
What attached file?
0
 

Author Comment

by:awalkinthepark
ID: 38724856
file attached
powerscript.txt
0
 
LVL 75

Accepted Solution

by:
käµfm³d   👽 earned 500 total points
ID: 38725094
Try:

using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;

namespace ConsoleApplication40
{
    class Program
    {
        static void Main(string[] args)
        {
            // Sample script to automate downloading of PDF images
            string[] header = { "item_id", "form_type", "sent_year", "co_name", "ein", "plan_number", "link", "facsimile_link" };
            string username = "YOUR_USER_NAME";
            string pass = "YOUR_PASSWORD";
            string codePageName = "UTF-8";
            string index_file_path = "2011-january.txt";
            string temp_file_path = "import.csv";
            string save_dir = "c:/temp";

            // read in an index.  Skip the first 5 lines as they contain other 
            // information and save it to a temporary file
            File.WriteAllLines(temp_file_path, File.ReadLines(index_file_path).Skip(5));

            // use import-csv to read our temp file and 
            // parse the listings within it.
            var list = File.ReadLines(temp_file_path)
                           .Select(line => line.Split('|'))
                           .Select(split => new
                                            {
                                                ack_id = split[0],
                                                form_type = split[1],
                                                filing_year = split[2],
                                                sponsor_name = split[3],
                                                ein = split[4],
                                                plan_number = split[5],
                                                link = split[6],
                                                facsimile_link = split[7],
                                            });

            // The site uses forms-based authentication.  
            // Therefore we need to first
            // login to the site using the login form page 
            // and retain the cookie
            // returned by the server for use later.

            string loginPostData = "userName=" + username + "&Password=" + pass;
            HttpWebRequest httpRequest = WebRequest.Create("http://mydomain.com/BulkFOIARequest/Account.aspx") as HttpWebRequest;
            httpRequest.AllowAutoRedirect = false;
            httpRequest.Method = "POST";
            httpRequest.ContentType = "application/x-www-form-urlencoded";

            Encoding enc = Encoding.GetEncoding(codePageName);
            byte[] bytes = enc.GetBytes(loginPostData);
            httpRequest.ContentLength = bytes.Length;
            Stream reqStream = httpRequest.GetRequestStream();
            reqStream.Write(bytes, 0, bytes.Length);
            reqStream.Flush();

            HttpWebResponse response = httpRequest.GetResponse() as HttpWebResponse;
            string cookie = response.Headers["Set-Cookie"]; // This is what we were after!

            // use WebClient for downloading individual pdfs for simplicity.
            WebClient webclient = new WebClient();
            webclient.Headers.Add("Cookie", " " + cookie.Replace("HttpOnly,", "HttpOnly; "));

            string filename = "";

            // enumerate through the list of filings and download each pdf image
            foreach (var entry in list)
            {
                // give each downloaded PDF a unique name
                if (entry.link.Length > 0)
                {
                    if (entry.item_id == "n/a")
                    {
                        MatchCollection matches = Regex.Matches(entry.link, @"dln=(\d{14})");
                        filename = matches[1] + ".pdf";
                    }
                    else
                    {
                        filename = entry.item_id + ".pdf";
                    }

                    webclient.DownloadFile(entry.link, filename);
                }

                if (entry.facsimile_link.Length > 0)
                {
                    filename = entry.item_id + "-facsimile.pdf";
                    filename = Path.Combine(save_dir, filename);
                    webclient.DownloadFile(entry.facsimile_link, filename);
                }
            }
        }
    }
}

Open in new window

0
 

Author Closing Comment

by:awalkinthepark
ID: 38725652
Works.  
Just modified it some  to fit our application, the guts of it are right.
Thanks!
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I thought I'd write this up for anyone who has a request to create an anonymous whistle-blower-type submission form created using SharePoint 2010 (this would probably work the same for 2013). It's not 100% fool-proof but it's as close as you can get…
A brief introduction to what I consider to be the best editor for PowerShell.
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
Although Jacob Bernoulli (1654-1705) has been credited as the creator of "Binomial Distribution Table", Gottfried Leibniz (1646-1716) did his dissertation on the subject in 1666; Leibniz you may recall is the co-inventor of "Calculus" and beat Isaac…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question