Download file from ftp

I need to automate the downloading of a data file from an ftp server to a network directory, then import the data into sql.   I have am using C# and using Filezilla.

I have tried:

1) I first looked at scripting the download in C# using this example:
http://stackoverflow.com/questions/2781654/ftpwebrequest-download-file

I believe I got error 550 or something.  I accidentally wiped out my test code..

2) The looked at doing the ftp from the command line using the examples here:
http://trac.filezilla-project.org/ticket/2317

I couldn't get anything I came up with to work.

3) So now I am looking at doing it in PowerShell.  Why, I don't know.  I found a script on internet.  I cannot get it to work either.

Below is my PS script and error.  I know little to nothing about ftp and less about PowerShell.    Good to use PS for this or another utility?  I'm open to anything.

Here's the PS script:
-------------------------------------------------
#ftp server
$ftp = "ftp://ftp.custsite.com
$user = "username"
$pass = "password"  

$ftpfile = "ftp://ftp.custsite.com\tx0072_2nd_mailing1.csv"

$localfile = "C:\Apps\mail1.csv"


$ftpclient = New-Object system.Net.WebClient

$uri = New-Object System.Uri($ftpfile)


$ftpclient.DownloadFile($uri,$localfile)



Here's the PSerror:
-----------------------------------
New-Object : Exception calling ".ctor" with "1" argument(s): "Invalid URI: The hostname could not be parsed."
At C:\apps\test.ps1.13 char:18

Exception calling "DownloadFile" with "2" arguments(): "Value cannot be null. Parameter name: address:
Delta7428Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

piattndCommented:
Make the following changes:

From:

$ftpfile = "ftp://ftp.custsite.com\tx0072_2nd_mailing1.csv"

to:

$ftpfile = "ftp://ftp.custsite.com/tx0072_2nd_mailing1.csv"

(Notice the "/" after the ".com") try that.
0
Delta7428Author Commented:
Now I am getting:

Exception calling "DownloadFile" with "2" argument(s): "The remote server returned an error: <530. Not logged in."

It is the same error I get if I log in to the site manually and enter the password incorrectly... I know the password I have in the script is correct.
0
piattndCommented:
When you browse to the site manually, is there a domain name or computer name listed that you're logging on to or any user@someplace.com username?  Are you using that full username?  If using a domain, are you using the "\" between the domain and the username instead of "/"?
0
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

piattndCommented:
Actually, you are never passing the login credentials in your FTP connection.  Notice how in code, you establish the username and password variables, but you never actually use them in your connection.

Try adding this connection after establishing your webclient:

$ftpclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass,"domain");

Open in new window


The domain section may not be needed, so remove it if you don't need it.  The final command without that would be:

$ftpclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Delta7428Author Commented:
This is what I see in the Filezilla header at the top when I log in manually:
 "User Defined Site Name - "username@ftp.ClientFTPSiteName.com
0
Delta7428Author Commented:
I changed:
         $ftp = "ftp://ftp.custsite.com

To:
         $ftp = "ftp://username@ftp.custsite.com"

I received the same error:
      Exception calling "DownloadFile" with "2" arguments(s): "The remote server returned an error: (530) Not logged in."

--------------
#ftp server
$ftp = "ftp://username@ftp.custsite.com"
$user = "username"
$pass = "password"  

$ftpfile = "ftp://ftp.custsite.com\tx0072_2nd_mailing1.csv"

$localfile = "C:\Apps\mail1.csv"


$ftpclient = New-Object system.Net.WebClient

$uri = New-Object System.Uri($ftpfile)


$ftpclient.DownloadFile($uri,$localfile)
0
piattndCommented:
If you're going to put the username and password in the URL to handle the authentication, you need to adjust your FTP variable to include it.

$ftp = "ftp://username:password@ftp.custsite.com"

Having said that, I wouldn't suggest it.  All someone would need to do is see your destination address and they would know your username and password for the site, even if you used SFTP.

If I were you, I'd do exactly as I said above and issue the credentials within the WebClient, not the actual URL, but it's up to you.
0
piattndCommented:
Did you get this working?
0
Delta7428Author Commented:
piattnd, I missed your posts prior to my last stupidest posts.

I had the site open for awhile, thought I refreshed it and checked for the new post before I postedposted.

I will look back at your post in the morning and see what I can do.  I'm juggling multiple  "#1" priorities..

Thank you.
0
piattndCommented:
No problem, just trying to get you the solution as fast as possible :)
0
Delta7428Author Commented:
ha pittnd, I hit submit on last post and my phone was flashing an email notifier from EE ... your last post just before my last post.  

You are psychic and tech expert.  kewl.

Get back to you soon.  Thanks for hanging.

And I totally agree with security concerns.  I had same.
0
Delta7428Author Commented:
Same happened again with the email flash on the phone on your last post.  :O

Thanks again.  I will try to get back to you first thing in the morning when I can re-avert my focus.
0
Delta7428Author Commented:
WooHoo! piattnd, You be too kewl for school!

Works!  Moved the file from the client site to ours.

I feel sure we will be moving the file, what if we just need to copy it and leave it on the client site?

-------------------------the worky code-------------------------
#ftp server
$ftp = "ftp://ftp.clientsite.com"
$user = "username"
$pass = "7s9417dm"  

$ftpfile = "ftp://ftp.clientsite.com/tx0072_2nd_mailing1.csv"

$localfile = "C:\Apps\mail1.csv"

$ftpclient = New-Object system.Net.WebClient
# $ftpclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass,"domain");

$ftpclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)

$uri = New-Object System.Uri($ftpfile)

$ftpclient.DownloadFile($uri,$localfile)
0
piattndCommented:
If you want to download, then delete the file, you'll need to use a different class.  By this, I'm referencing the "System.Net.WebClient" as a class.  The "System.Net.WebClient" does not have a method for handling delete requests, but the "System.Net.FtpWebRequest" does.

If you're going to be including the removal of the file from the client site, you're going to want to make sure you check the file you downloaded is the exact same file you're deleting, meaning you didn't only download part of the file.  If you still want to pursue adding in all this logic and removing the file, try this link out.  It's not in powershell, but using the classes works the same way.  Let me elaborate:

Link to MSDN Article:
http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest.aspx

public static bool DeleteFileOnServer(Uri serverUri)
{
    // The serverUri parameter should use the ftp:// scheme. 
    // It contains the name of the server file that is to be deleted. 
    // Example: ftp://contoso.com/someFile.txt. 
    //  

    if (serverUri.Scheme != Uri.UriSchemeFtp)
    {
        return false;
    }
    // Get the object used to communicate with the server.
    FtpWebRequest request = (FtpWebRequest)WebRequest.Create(serverUri);
    request.Method = WebRequestMethods.Ftp.DeleteFile;

    FtpWebResponse response = (FtpWebResponse) request.GetResponse();
    Console.WriteLine("Delete status: {0}",response.StatusDescription);  
    response.Close();
    return true;
}

Open in new window


That is the code in C#, so for powershell, check this document out:

http://social.technet.microsoft.com/Forums/windowsserver/en-US/2e07df8a-c7cd-47be-8e13-35de9b9189a8/remove-files-on-ftp-site-with-powershell
0
Delta7428Author Commented:
SI was mistaken when I said "Moved" the file and bad choice of words.

After script ran, I took at quick look at the local destination location and verified it was there.

I didn't realize my filezilla session had timed out and the file(s) that I saw previously .. only a couple were not showing on the site.  So I initially thought it was "moved" .. deleted  from their site.

Script works.  File copied as intended.  Original is file still on client ftp.

Got what I need.

I will take a look at documentation you provided.

Points coming.  Thanks!
0
Delta7428Author Commented:
Very quick response and follow through!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.