Solved

Best FTP file transfter

Posted on 2010-11-09
8
1,202 Views
Last Modified: 2012-05-10
Dear Expert,

Now I am using pscp.exe software to upload 60 files to my website and I am doing benchmarking
which option is faster to ftp those files, I switch  the option  
-4 -6 force use of IPv4 or IPV6
-sftp force use SFTP protocol
-scp force use of SCP protocol

All option doesn't change the speed too much,

And I try to use ftpputfile API function in VBA , the speed is faster if  increase buffer_size from 4k to 65K
but the VBA is single tasking  and could not do mutlitasking or pipeling to send 50 files in 50 processes.

Now I am going to open 50 batch files to create 50 process using pscp.exe program that help a lot and almost
reduce 2/3 upload time from new sub of speedtest() as follows. The speed is now better but still slow. Please help
to reply my following question

Question-1 Try want to know which method in
the market is the best for ftp file transfer besides using pspc.exe  ?
Question-2 Which FTP protocoll is fastest ?
Question-3, COuld I do zip all 50 files in Windows Vista  int one and ftp it to server but how I
unzip the one file in my website and done unzip  operation by my index.htm in my /public_html or javasript code. ?
 

Please advise
Duncan
Sub speedtest()
dim i as integer
Do until i=51
shell("cmd.exe /c" & "pscp" & i & ".bat",2)
i=i+1
loop
End test


 
 
0
Comment
Question by:duncanb7
8 Comments
 
LVL 15

Expert Comment

by:JBond2010
ID: 34093127
CoreFTP is not too bad and this is free software. I have used this for tranferring websites on databases etc.
0
 
LVL 2

Accepted Solution

by:
Hossy earned 167 total points
ID: 34093144
Obviously you're on the right track with #3: In whatever FTP transfer method you choose, one that employs compression (since you're using highly compressible files), is going to be your best bet.

Now, time-wise, an FTP utility that first ZIPs the files locally before uploading the ZIP file is going to have an impact on the transfer time since it compresses first then transfers.  You're looking for speed, so I'd look for an FTP utility that compresses on-the-fly.

This article, http://www.experts-exchange.com/Networking/Misc/Q_20939305.html, discusses this.  However, in most all cases, you will need to install a program and run a script on your website's server so it knows how to do its part to assist in a speedy file transfer.
0
 
LVL 10

Expert Comment

by:moon_blue69
ID: 34093152
filezilla is another option free and cool
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 5

Assisted Solution

by:Leithauser
Leithauser earned 333 total points
ID: 34095048
Here is complete code for an FTP transfer. I cannot say whether it is faster or slower than the one you use, but give it a try and see.

Option Explicit

Private Type FILETIME
  dwLowDateTime As Long
  dwHighDateTime As Long
End Type

Private Type WIN32_FIND_DATA
  dwFileAttributes As Long
  ftCreationTime As FILETIME
  ftLastAccessTime As FILETIME
  ftLastWriteTime As FILETIME
  nFileSizeHigh As Long
  nFileSizeLow As Long
  dwReserved0 As Long
  dwReserved1 As Long
  cFileName As String * 50 ' MAX_PATH
  cAlternate As String * 14
End Type

Public Declare Function FtpSetCurrentDirectory Lib "wininet.dll" Alias "FtpSetCurrentDirectoryA" _
    (ByVal hFtpSession As Long, ByVal lpszDirectory As String) As Boolean

Private Declare Function InternetOpen Lib "wininet.dll" Alias "InternetOpenA" _
    (ByVal sAgent As String, ByVal lAccessType As Long, ByVal sProxyName As String, _
    ByVal sProxyBypass As String, ByVal lFlags As Long) As Long
'The sAgent parameter is used to specify the application or entity calling the WinINet functions. For our purposes, we can put "MyFTP Control".
'The lAccessType parameter specifies whether we connect directly to a host or whether we use a proxy server for the connection. If we pass the value 1, we'll connect directly to the host. If we pass the value 3, we'll connect via a proxy server. If we pass 0, we'll connect based on the registry values ProxyEnable, ProxyServer, and ProxyOverride located under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings.
'Rather than using the registry settings, we can use the sProxyName and sProxyBypass parameters to provide the proxy server (or servers) and the list of IP addresses or names that should not be routed to the proxy. The basic format for listing a proxy is "protocol=protocol://proxy_name:access_port". For example, to specify port 21 on Proxy1 as the proxy server, use "ftp=ftp://Proxy1:21" as the sProxyName. To bypass any host that starts with "nov", the sProxyBypass string would be "nov*".
'Finally, lFlags is used to indicate various options affecting the behavior of the InternetOpen function. For our example, we'll pass 0.
'So, to open an Internet session without using a proxy, our call would be like this:
'lngINet = InternetOpen("MyFTP Control", 1, vbNullString, vbNullString, 0)
'If the function call fails, lngINet will be 0. Otherwise, lngINet holds the value of the handle that we'll need to pass to the InternetConnect function in the next step.

Private Declare Function InternetConnect Lib "wininet.dll" Alias "InternetConnectA" _
    (ByVal hInternetSession As Long, ByVal sServerName As String, _
    ByVal nServerPort As Integer, ByVal sUsername As String, _
    ByVal sPassword As String, ByVal lService As Long, _
    ByVal lFlags As Long, ByVal lContext As Long) As Long
'The first parameter, hInternetSession is the value of the handle returned by the InternetOpen call.
'sServerName is the IP address or host name of the FTP server we're connecting to.
'nServerPort indicates which port to connect to. For our example, we'll use the value 0, which indicates the default port (21).
'sUsername and sPassword are used to pass the user name and password, respectively.
'The lService parameter is used to indicate the type of service to access, i.e., HTTP, FTP, etc. We'll always pass the value of 1, indicating the FTP service.
'If we pass the value x8000000 in the lFlags parameter, the connection will use passive FTP semantics. Otherwise, as in our example, we can pass 0 to use non-passive semantics.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to connect to the host FTP.MICROSOFT.COM using the anonymous userid:
'lngINetConn = InternetConnect(lngINet, "ftp.microsoft.com", 0, _
    "anonymous", "wally@wallyworld.com", 1, 0, 0)
'If the function call fails, lngINetConn will be 0. Otherwise, lngINetConn holds the value of the handle that we'll need to pass to the FtpGetFile function in the next step.

Private Declare Function FtpGetFile Lib "wininet.dll" Alias "FtpGetFileA" _
    (ByVal hFtpSession As Long, ByVal lpszRemoteFile As String, _
    ByVal lpszNewFile As String, ByVal fFailIfExists As Boolean, _
    ByVal dwFlagsAndAttributes As Long, ByVal dwFlags As Long, _
    ByVal dwContext As Long) As Boolean
'The first parameter, hFtpSession is the value of the handle returned by the InternetConnect call.
'lpszRemoteFile and lpszNewFile are the names of the file on the FTP server and the file to create on the local machine, respectively.
'The fFailIfExists flag is either 0 (replace local file) or -1 (fail if local file already exists).
'dwFlagsAndAttributes can be used to specify file attributes for the local file. For our example, we'll overlook this and just pass 0.
'We use the dwFlags parameter to specify 1 for transferring the file in ASCII (Type A transfer method) or 2 for transferring the file in Binary (Type I transfer method). Since DIRMAP.TXT is an ASCII text file, we'll pass the value of 1.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to get the DIRMAP.TXT file and store it in C:\DIRMAP.TXT. In case the local file already exists, we'll overwrite it.
'blnRC = FtpGetFile(lngINetConn, "dirmap.txt", "c:\dirmap.txt", 0, 0, 1, 0)
'If the function call is successful, blnRC will be True, otherwise, blnRC will be False.

Private Declare Function InternetCloseHandle Lib "wininet.dll" (ByVal hInet As Long) As Integer

Private Declare Function FtpPutFile Lib "wininet.dll" Alias "FtpPutFileA" _
    (ByVal hFtpSession As Long, ByVal lpszLocalFile As String, _
    ByVal lpszRemoteFile As String, ByVal dwFlags As Long, _
    ByVal dwContext As Long) As Boolean
'The first parameter, hFtpSession is the value of the handle returned by the InternetConnect call.
'lpszNewFile and lpszRemoteFile are the names of the file on the local machine and the file to create on the remote host, respectively.
'We use the dwFlags parameter to specify 1 for transferring the file in ASCII (Type A transfer method) or 2 for transferring the file in Binary (Type I transfer method). Since DIRMAP.TXT is an ASCII text file, we'll pass the value of 1.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to get the C:\DIRMAP.TXT file and store it as DIRMAP.TXT on an FTP server:
'blnRC = FtpPutFile(lngINetConn, "c:\dirmap.txt", "dirmap.txt", 1, 0)

Private Declare Function FtpDeleteFile Lib "wininet.dll" Alias "FtpDeleteFileA" _
    (ByVal hFtpSession As Long, ByVal lpszFileName As String) As Boolean

Private Declare Function FtpFindFirstFile Lib "wininet.dll" Alias "FtpFindFirstFileA" (ByVal hFtpSession As Long, ByVal lpszSearchFile As String, lpFindFileData As WIN32_FIND_DATA, ByVal dwFlags As Long, ByVal dwContent As Long) As Long

Private Declare Function InternetFindNextFile Lib "wininet.dll" Alias "InternetFindNextFileA" _
    (ByVal hFind As Long, lpvFindData As WIN32_FIND_DATA) As Long

Sub DownloadFileByFTP(LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String)
  'FileType=1 for ASCII, 2 for binary
  Dim lngINet As Long
  Dim sUsername As String
  Dim blnRC As Boolean
  If Len(FTPUserName) Then
    sUsername = FTPUserName
  Else
    sUsername = "anonymous"
  End If
  lngINet = InternetOpen("CRCFTP Control", 1, vbNullString, vbNullString, 0)
  If lngINet = 0 Then
    lngINet = InternetOpen("CRCFTP Control", 0, vbNullString, vbNullString, 0)
  End If
  If lngINet = 0 Then
    DirectFTPForm.lblStatus = "Step 1 failed." ' "Internet connection failed.", "Step 1", 0
    Exit Sub
  End If
  lngINetConn = InternetConnect(lngINet, FTPAddress, 0, sUsername, FTPPassword, 1, 0, 0)
  If lngINetConn = 0 Then
    DirectFTPForm.lblStatus = "Step 2 failed."  '"Internet connection failed.", "Step 2", 0
    Exit Sub
  End If
  If Len(FolderName) Then
    Dim bRet As Boolean, sPathFromRoot As String
    sPathFromRoot = "/" & FolderName
    bRet = FtpSetCurrentDirectory(lngINetConn, sPathFromRoot)
    If bRet = False Then DirectFTPForm.lblStatus = Err.LastDllError & "in rcd"
  End If
  blnRC = FtpGetFile(lngINetConn, DestinationFile, LocalFile, 0, 0, FileType, 0)
  'blnRC = FtpPutFile(lngINetConn, LocalFile, DestinationFile, 1, 0)
  If blnRC = False Then DirectFTPForm.lblStatus = "Did not work." 'MsgBox "Did not work.", , "Problem"
  InternetCloseHandle lngINetConn
  InternetCloseHandle lngINet
End Sub

Sub UploadFileByFTP(LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String)
  'FileType=1 for ASCII, 2 for binary
  Dim lngINet As Long
  Dim sUsername As String
  Dim blnRC As Boolean
  If Len(FTPUserName) Then
    sUsername = FTPUserName
  Else
    sUsername = "anonymous"
  End If
  lngINet = InternetOpen("CRCFTP Control", 1, vbNullString, vbNullString, 0)
  If lngINet = 0 Then
    lngINet = InternetOpen("CRCFTP Control", 0, vbNullString, vbNullString, 0)
  End If
  If lngINet = 0 Then
    DirectFTPForm.lblStatus = "Step 1 failed"
    'MsgBox "lblStatus", 0
    Exit Sub
  Else
    DirectFTPForm.lblStatus = "Step 1 done"
  End If
  DoEvents
  lngINetConn = InternetConnect(lngINet, FTPAddress, 0, sUsername, FTPPassword, 1, 0, 0)
  If lngINetConn = 0 Then
    'MsgBox "Internet connection failed.", , "Step 2"
    DirectFTPForm.lblStatus = "Step 2 failed"
    Exit Sub
  Else
    DirectFTPForm.lblStatus = "Step 2 done"
  End If
  DoEvents
  If Len(FolderName) Then
    Dim bRet As Boolean, sPathFromRoot As String
    sPathFromRoot = "/" & FolderName
    bRet = FtpSetCurrentDirectory(lngINetConn, sPathFromRoot)
    If bRet = False Then
      DirectFTPForm.lblStatus = Err.LastDllError & "Error changing directory"
      InternetCloseHandle lngINetConn
      InternetCloseHandle lngINet
      Exit Sub
    End If
  End If
  blnRC = FtpPutFile(lngINetConn, LocalFile, DestinationFile, FileType, 0)
  If blnRC = False Then
    DirectFTPForm.lblStatus = "Did not work."
  Else
    DirectFTPForm.lblStatus = "Success."
  End If
  
  DoEvents
  InternetCloseHandle lngINetConn
  InternetCloseHandle lngINet
End Sub
Private Sub rcd(pszDir As String)
  If pszDir = "" Then
    MsgBox "Please enter the directory to CD"
    Exit Sub
  Else
    Dim sPathFromRoot As String
    Dim bRet As Boolean
    'If InStr(1, pszDir, txtServer.Text) Then
    '    sPathFromRoot = Mid(pszDir, Len(txtServer.Text) + 1, Len(pszDir) - Len(txtServer.Text))
    'Else
        sPathFromRoot = "/" & pszDir
    'End If
    'If sPathFromRoot = "" Then sPathFromRoot = "/"
    bRet = FtpSetCurrentDirectory(hConnection, sPathFromRoot)
    If bRet = False Then DirectFTPForm.lblStatus = Err.LastDllError & "in rcd"
  End If
End Sub

'Here is some code to call it. If you replace the text boxes with the obvious values they should contain, it should do the transfer.

Private Sub Command1_Click()
  Dim LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String
  LocalFile = Text2.Text
  DestinationFile = Text3.Text
  FTPAddress = Text1.Text
  FTPUserName = Text4.Text
  FTPPassword = Text5.Text
  FolderName = Text6.Text
  If Option1 Then FileType = 1 ' ASCII
  If Option2 Then FileType = 2 ' Binary
  If FileType = 0 Then
      MsgBox "You must select a file type.", , "Error"
  End If
  
  UploadFileByFTP LocalFile, DestinationFile, FTPAddress, FTPUserName, FTPPassword, FileType, FolderName

End Sub

Open in new window

0
 
LVL 13

Author Comment

by:duncanb7
ID: 34097739
Yes, thanks for your code, and it is time for deep VB training, I will try it and target to
reduce upload time by 10

0
 
LVL 5

Assisted Solution

by:Leithauser
Leithauser earned 333 total points
ID: 34103117
<<Yes, thanks for your code, and it is time for deep VB training, I will try it and target to
reduce upload time by 10>>
I do not really know whether the code I gave you is particularly fast, but you should be able to cut and paste everything down to "Here is some code to call it." into a single code module. You can then replace

LocalFile = Text2.Text
DestinationFile = Text3.Text
FTPAddress = Text1.Text
FTPUserName = Text4.Text
FTPPassword = Text5.Text
FolderName = Text6.Text

with straight code like
LocalFile = "C:\MyFiles\abc.html"
to make the code work.
You need to set FileType to either 1 or 2, and I suggest that you experiment with which one works best, assuming that your files are ASCII and that will work. (I assume that 1 would be fastest if your files are ASCII.) If they are binary, you need to set FileType to 2.

0
 
LVL 13

Author Closing Comment

by:duncanb7
ID: 34220116
Thank for your reply and
it is getting better on speed
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now