Solved

Best FTP file transfter

Posted on 2010-11-09
8
1,225 Views
Last Modified: 2012-05-10
Dear Expert,

Now I am using pscp.exe software to upload 60 files to my website and I am doing benchmarking
which option is faster to ftp those files, I switch  the option  
-4 -6 force use of IPv4 or IPV6
-sftp force use SFTP protocol
-scp force use of SCP protocol

All option doesn't change the speed too much,

And I try to use ftpputfile API function in VBA , the speed is faster if  increase buffer_size from 4k to 65K
but the VBA is single tasking  and could not do mutlitasking or pipeling to send 50 files in 50 processes.

Now I am going to open 50 batch files to create 50 process using pscp.exe program that help a lot and almost
reduce 2/3 upload time from new sub of speedtest() as follows. The speed is now better but still slow. Please help
to reply my following question

Question-1 Try want to know which method in
the market is the best for ftp file transfer besides using pspc.exe  ?
Question-2 Which FTP protocoll is fastest ?
Question-3, COuld I do zip all 50 files in Windows Vista  int one and ftp it to server but how I
unzip the one file in my website and done unzip  operation by my index.htm in my /public_html or javasript code. ?
 

Please advise
Duncan
Sub speedtest()
dim i as integer
Do until i=51
shell("cmd.exe /c" & "pscp" & i & ".bat",2)
i=i+1
loop
End test


 
 
0
Comment
Question by:duncanb7
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 15

Expert Comment

by:JBond2010
ID: 34093127
CoreFTP is not too bad and this is free software. I have used this for tranferring websites on databases etc.
0
 
LVL 2

Accepted Solution

by:
Hossy earned 167 total points
ID: 34093144
Obviously you're on the right track with #3: In whatever FTP transfer method you choose, one that employs compression (since you're using highly compressible files), is going to be your best bet.

Now, time-wise, an FTP utility that first ZIPs the files locally before uploading the ZIP file is going to have an impact on the transfer time since it compresses first then transfers.  You're looking for speed, so I'd look for an FTP utility that compresses on-the-fly.

This article, http://www.experts-exchange.com/Networking/Misc/Q_20939305.html, discusses this.  However, in most all cases, you will need to install a program and run a script on your website's server so it knows how to do its part to assist in a speedy file transfer.
0
 
LVL 10

Expert Comment

by:moon_blue69
ID: 34093152
filezilla is another option free and cool
0
Free NetCrunch network monitor licenses!

Only on Experts-Exchange: Sign-up for a free-trial and we'll send you your permanent license!

Here is what you get: 30 Nodes | Unlimited Sensors | No Time Restrictions | Absolutely FREE!

Act now. This offer ends July 14, 2017.

 
LVL 5

Assisted Solution

by:Leithauser
Leithauser earned 333 total points
ID: 34095048
Here is complete code for an FTP transfer. I cannot say whether it is faster or slower than the one you use, but give it a try and see.

Option Explicit

Private Type FILETIME
  dwLowDateTime As Long
  dwHighDateTime As Long
End Type

Private Type WIN32_FIND_DATA
  dwFileAttributes As Long
  ftCreationTime As FILETIME
  ftLastAccessTime As FILETIME
  ftLastWriteTime As FILETIME
  nFileSizeHigh As Long
  nFileSizeLow As Long
  dwReserved0 As Long
  dwReserved1 As Long
  cFileName As String * 50 ' MAX_PATH
  cAlternate As String * 14
End Type

Public Declare Function FtpSetCurrentDirectory Lib "wininet.dll" Alias "FtpSetCurrentDirectoryA" _
    (ByVal hFtpSession As Long, ByVal lpszDirectory As String) As Boolean

Private Declare Function InternetOpen Lib "wininet.dll" Alias "InternetOpenA" _
    (ByVal sAgent As String, ByVal lAccessType As Long, ByVal sProxyName As String, _
    ByVal sProxyBypass As String, ByVal lFlags As Long) As Long
'The sAgent parameter is used to specify the application or entity calling the WinINet functions. For our purposes, we can put "MyFTP Control".
'The lAccessType parameter specifies whether we connect directly to a host or whether we use a proxy server for the connection. If we pass the value 1, we'll connect directly to the host. If we pass the value 3, we'll connect via a proxy server. If we pass 0, we'll connect based on the registry values ProxyEnable, ProxyServer, and ProxyOverride located under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings.
'Rather than using the registry settings, we can use the sProxyName and sProxyBypass parameters to provide the proxy server (or servers) and the list of IP addresses or names that should not be routed to the proxy. The basic format for listing a proxy is "protocol=protocol://proxy_name:access_port". For example, to specify port 21 on Proxy1 as the proxy server, use "ftp=ftp://Proxy1:21" as the sProxyName. To bypass any host that starts with "nov", the sProxyBypass string would be "nov*".
'Finally, lFlags is used to indicate various options affecting the behavior of the InternetOpen function. For our example, we'll pass 0.
'So, to open an Internet session without using a proxy, our call would be like this:
'lngINet = InternetOpen("MyFTP Control", 1, vbNullString, vbNullString, 0)
'If the function call fails, lngINet will be 0. Otherwise, lngINet holds the value of the handle that we'll need to pass to the InternetConnect function in the next step.

Private Declare Function InternetConnect Lib "wininet.dll" Alias "InternetConnectA" _
    (ByVal hInternetSession As Long, ByVal sServerName As String, _
    ByVal nServerPort As Integer, ByVal sUsername As String, _
    ByVal sPassword As String, ByVal lService As Long, _
    ByVal lFlags As Long, ByVal lContext As Long) As Long
'The first parameter, hInternetSession is the value of the handle returned by the InternetOpen call.
'sServerName is the IP address or host name of the FTP server we're connecting to.
'nServerPort indicates which port to connect to. For our example, we'll use the value 0, which indicates the default port (21).
'sUsername and sPassword are used to pass the user name and password, respectively.
'The lService parameter is used to indicate the type of service to access, i.e., HTTP, FTP, etc. We'll always pass the value of 1, indicating the FTP service.
'If we pass the value x8000000 in the lFlags parameter, the connection will use passive FTP semantics. Otherwise, as in our example, we can pass 0 to use non-passive semantics.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to connect to the host FTP.MICROSOFT.COM using the anonymous userid:
'lngINetConn = InternetConnect(lngINet, "ftp.microsoft.com", 0, _
    "anonymous", "wally@wallyworld.com", 1, 0, 0)
'If the function call fails, lngINetConn will be 0. Otherwise, lngINetConn holds the value of the handle that we'll need to pass to the FtpGetFile function in the next step.

Private Declare Function FtpGetFile Lib "wininet.dll" Alias "FtpGetFileA" _
    (ByVal hFtpSession As Long, ByVal lpszRemoteFile As String, _
    ByVal lpszNewFile As String, ByVal fFailIfExists As Boolean, _
    ByVal dwFlagsAndAttributes As Long, ByVal dwFlags As Long, _
    ByVal dwContext As Long) As Boolean
'The first parameter, hFtpSession is the value of the handle returned by the InternetConnect call.
'lpszRemoteFile and lpszNewFile are the names of the file on the FTP server and the file to create on the local machine, respectively.
'The fFailIfExists flag is either 0 (replace local file) or -1 (fail if local file already exists).
'dwFlagsAndAttributes can be used to specify file attributes for the local file. For our example, we'll overlook this and just pass 0.
'We use the dwFlags parameter to specify 1 for transferring the file in ASCII (Type A transfer method) or 2 for transferring the file in Binary (Type I transfer method). Since DIRMAP.TXT is an ASCII text file, we'll pass the value of 1.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to get the DIRMAP.TXT file and store it in C:\DIRMAP.TXT. In case the local file already exists, we'll overwrite it.
'blnRC = FtpGetFile(lngINetConn, "dirmap.txt", "c:\dirmap.txt", 0, 0, 1, 0)
'If the function call is successful, blnRC will be True, otherwise, blnRC will be False.

Private Declare Function InternetCloseHandle Lib "wininet.dll" (ByVal hInet As Long) As Integer

Private Declare Function FtpPutFile Lib "wininet.dll" Alias "FtpPutFileA" _
    (ByVal hFtpSession As Long, ByVal lpszLocalFile As String, _
    ByVal lpszRemoteFile As String, ByVal dwFlags As Long, _
    ByVal dwContext As Long) As Boolean
'The first parameter, hFtpSession is the value of the handle returned by the InternetConnect call.
'lpszNewFile and lpszRemoteFile are the names of the file on the local machine and the file to create on the remote host, respectively.
'We use the dwFlags parameter to specify 1 for transferring the file in ASCII (Type A transfer method) or 2 for transferring the file in Binary (Type I transfer method). Since DIRMAP.TXT is an ASCII text file, we'll pass the value of 1.
'Finally, lContext is used to identify the application context when using callbacks. Since we're not using callbacks for our example, we'll pass 0.
'So here's the call to get the C:\DIRMAP.TXT file and store it as DIRMAP.TXT on an FTP server:
'blnRC = FtpPutFile(lngINetConn, "c:\dirmap.txt", "dirmap.txt", 1, 0)

Private Declare Function FtpDeleteFile Lib "wininet.dll" Alias "FtpDeleteFileA" _
    (ByVal hFtpSession As Long, ByVal lpszFileName As String) As Boolean

Private Declare Function FtpFindFirstFile Lib "wininet.dll" Alias "FtpFindFirstFileA" (ByVal hFtpSession As Long, ByVal lpszSearchFile As String, lpFindFileData As WIN32_FIND_DATA, ByVal dwFlags As Long, ByVal dwContent As Long) As Long

Private Declare Function InternetFindNextFile Lib "wininet.dll" Alias "InternetFindNextFileA" _
    (ByVal hFind As Long, lpvFindData As WIN32_FIND_DATA) As Long

Sub DownloadFileByFTP(LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String)
  'FileType=1 for ASCII, 2 for binary
  Dim lngINet As Long
  Dim sUsername As String
  Dim blnRC As Boolean
  If Len(FTPUserName) Then
    sUsername = FTPUserName
  Else
    sUsername = "anonymous"
  End If
  lngINet = InternetOpen("CRCFTP Control", 1, vbNullString, vbNullString, 0)
  If lngINet = 0 Then
    lngINet = InternetOpen("CRCFTP Control", 0, vbNullString, vbNullString, 0)
  End If
  If lngINet = 0 Then
    DirectFTPForm.lblStatus = "Step 1 failed." ' "Internet connection failed.", "Step 1", 0
    Exit Sub
  End If
  lngINetConn = InternetConnect(lngINet, FTPAddress, 0, sUsername, FTPPassword, 1, 0, 0)
  If lngINetConn = 0 Then
    DirectFTPForm.lblStatus = "Step 2 failed."  '"Internet connection failed.", "Step 2", 0
    Exit Sub
  End If
  If Len(FolderName) Then
    Dim bRet As Boolean, sPathFromRoot As String
    sPathFromRoot = "/" & FolderName
    bRet = FtpSetCurrentDirectory(lngINetConn, sPathFromRoot)
    If bRet = False Then DirectFTPForm.lblStatus = Err.LastDllError & "in rcd"
  End If
  blnRC = FtpGetFile(lngINetConn, DestinationFile, LocalFile, 0, 0, FileType, 0)
  'blnRC = FtpPutFile(lngINetConn, LocalFile, DestinationFile, 1, 0)
  If blnRC = False Then DirectFTPForm.lblStatus = "Did not work." 'MsgBox "Did not work.", , "Problem"
  InternetCloseHandle lngINetConn
  InternetCloseHandle lngINet
End Sub

Sub UploadFileByFTP(LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String)
  'FileType=1 for ASCII, 2 for binary
  Dim lngINet As Long
  Dim sUsername As String
  Dim blnRC As Boolean
  If Len(FTPUserName) Then
    sUsername = FTPUserName
  Else
    sUsername = "anonymous"
  End If
  lngINet = InternetOpen("CRCFTP Control", 1, vbNullString, vbNullString, 0)
  If lngINet = 0 Then
    lngINet = InternetOpen("CRCFTP Control", 0, vbNullString, vbNullString, 0)
  End If
  If lngINet = 0 Then
    DirectFTPForm.lblStatus = "Step 1 failed"
    'MsgBox "lblStatus", 0
    Exit Sub
  Else
    DirectFTPForm.lblStatus = "Step 1 done"
  End If
  DoEvents
  lngINetConn = InternetConnect(lngINet, FTPAddress, 0, sUsername, FTPPassword, 1, 0, 0)
  If lngINetConn = 0 Then
    'MsgBox "Internet connection failed.", , "Step 2"
    DirectFTPForm.lblStatus = "Step 2 failed"
    Exit Sub
  Else
    DirectFTPForm.lblStatus = "Step 2 done"
  End If
  DoEvents
  If Len(FolderName) Then
    Dim bRet As Boolean, sPathFromRoot As String
    sPathFromRoot = "/" & FolderName
    bRet = FtpSetCurrentDirectory(lngINetConn, sPathFromRoot)
    If bRet = False Then
      DirectFTPForm.lblStatus = Err.LastDllError & "Error changing directory"
      InternetCloseHandle lngINetConn
      InternetCloseHandle lngINet
      Exit Sub
    End If
  End If
  blnRC = FtpPutFile(lngINetConn, LocalFile, DestinationFile, FileType, 0)
  If blnRC = False Then
    DirectFTPForm.lblStatus = "Did not work."
  Else
    DirectFTPForm.lblStatus = "Success."
  End If
  
  DoEvents
  InternetCloseHandle lngINetConn
  InternetCloseHandle lngINet
End Sub
Private Sub rcd(pszDir As String)
  If pszDir = "" Then
    MsgBox "Please enter the directory to CD"
    Exit Sub
  Else
    Dim sPathFromRoot As String
    Dim bRet As Boolean
    'If InStr(1, pszDir, txtServer.Text) Then
    '    sPathFromRoot = Mid(pszDir, Len(txtServer.Text) + 1, Len(pszDir) - Len(txtServer.Text))
    'Else
        sPathFromRoot = "/" & pszDir
    'End If
    'If sPathFromRoot = "" Then sPathFromRoot = "/"
    bRet = FtpSetCurrentDirectory(hConnection, sPathFromRoot)
    If bRet = False Then DirectFTPForm.lblStatus = Err.LastDllError & "in rcd"
  End If
End Sub

'Here is some code to call it. If you replace the text boxes with the obvious values they should contain, it should do the transfer.

Private Sub Command1_Click()
  Dim LocalFile As String, DestinationFile As String, FTPAddress As String, FTPUserName As String, FTPPassword As String, FileType As Long, FolderName As String
  LocalFile = Text2.Text
  DestinationFile = Text3.Text
  FTPAddress = Text1.Text
  FTPUserName = Text4.Text
  FTPPassword = Text5.Text
  FolderName = Text6.Text
  If Option1 Then FileType = 1 ' ASCII
  If Option2 Then FileType = 2 ' Binary
  If FileType = 0 Then
      MsgBox "You must select a file type.", , "Error"
  End If
  
  UploadFileByFTP LocalFile, DestinationFile, FTPAddress, FTPUserName, FTPPassword, FileType, FolderName

End Sub

Open in new window

0
 
LVL 13

Author Comment

by:duncanb7
ID: 34097739
Yes, thanks for your code, and it is time for deep VB training, I will try it and target to
reduce upload time by 10

0
 
LVL 5

Assisted Solution

by:Leithauser
Leithauser earned 333 total points
ID: 34103117
<<Yes, thanks for your code, and it is time for deep VB training, I will try it and target to
reduce upload time by 10>>
I do not really know whether the code I gave you is particularly fast, but you should be able to cut and paste everything down to "Here is some code to call it." into a single code module. You can then replace

LocalFile = Text2.Text
DestinationFile = Text3.Text
FTPAddress = Text1.Text
FTPUserName = Text4.Text
FTPPassword = Text5.Text
FolderName = Text6.Text

with straight code like
LocalFile = "C:\MyFiles\abc.html"
to make the code work.
You need to set FileType to either 1 or 2, and I suggest that you experiment with which one works best, assuming that your files are ASCII and that will work. (I assume that 1 would be fastest if your files are ASCII.) If they are binary, you need to set FileType to 2.

0
 
LVL 13

Author Closing Comment

by:duncanb7
ID: 34220116
Thank for your reply and
it is getting better on speed
0

Featured Post

SharePoint Admin?

Enable Your Employees To Focus On The Core With Intuitive Onscreen Guidance That is With You At The Moment of Need.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When trying to find the cause of a problem in VBA or VB6 it's often valuable to know what procedures were executed prior to the error. You can use the Call Stack for that but it is often inadequate because it may show procedures you aren't interesteā€¦
I was working on a PowerPoint add-in the other day and a client asked me "can you implement a feature which processes a chart when it's pasted into a slide from another deck?". It got me wondering how to hook into built-in ribbon events in Office.
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applicā€¦
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question