Solved

FTP vs copy command

Posted on 2002-03-07
14
2,847 Views
Last Modified: 2008-01-09
Hi,

I'm curious which is faster for performing the task?  If I were to copy a file from one machine to another machine both connected on the same LAN or WAN, is FTP faster or is a straight copy command faster?  

The operating system would be a Windows based system, primarily either Windows NT Server or Windows 2000 Server.  Either type of operating system would reside on either machines but both would be some flavor of Windows.  

Any assistance would greatly be apprecriated.

Thanks
0
Comment
Question by:davism
14 Comments
 
LVL 12

Expert Comment

by:pjknibbs
Comment Utility
Neither can be done directly. The COPY command won't accept UNC pathnames, so you would first have to map a network drive using NET USE before you could carry out your copy. With FTP you obviously need to have an FTP server configured at the other end of the connection in order to perform the operation.

With that said, I'd be inclined to say that FTP would probably be faster--it has less overhead than the normal NETBIOS calls used for the COPY command.
0
 
LVL 1

Author Comment

by:davism
Comment Utility
I don't understand what you just said about the COPY commend wont accept UNC pathnames.  COPY command does accept UNC specifications thereby NOT having to use a NET USE.

I use a COPY command with UNC specifications on a daily basis.

For example:

c:\>copy "\\{source_machine_name}\{sharename}\{directory_spec}\{filename}" "\\{destination_machine_name}\{sharename}\{directory_spec}\{filename}"

That being said - how much of a factor is that in whether which is faster?
0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
I oft opt for different approach, to expand beyond the moment.

For LAN, use copy, or one of its flavors.
For WAN, use ftp if internet.

FTP is generally more reliable (when secured, which can take more work)

Unlike ftp, copy preserves files datestamp info, and over time I've found this rather critical to distinguish one file or download from another. A workaround of sorts is to ftp a package, zipped up. The unzipped package will preserve the datestamp, but the .zip file won't.

For actual transmit time, one has to also consider how a three-way process can impact. You've got origination (original), destination (the final location), and at least the source (client?) where the initial copy command is being run. There could be (and are) intermediate devices that can impact (including packet sizings).

> I use a COPY command with UNC specifications on a daily basis.

When having specific need for specific application, the answer is best obtained by simply running the two and comparing.  Where the result is <1 sec. say one method is 2 msec faster, it's like "who cares" (depends on how many thousands of times run per day, perhaps)

> c:\>copy "\\{source_machine....

This has appearance of method that has file move from origination to client to destination, hence twice the packets of situation where the client is either the source or destination. Thus it should be found as slower, all else equal. That is reference:

> For LAN, use copy, or one of its flavors.

I believe there are a number of alternatives to copy that are otherwise similar, that do not need the intermediary, way station.  "ncopy" comes to mind as being among them. But my memory fades here. The 'n' may mean network-copy. And I think DOS alternatives, and Netware had versions of copy that were better, at networking, but that xCopy did not have the networking improvement capability. Whether Win2k or other upgrade changed anything on that, I dunno, but MS history leads one to exect no change.

Answer IMO is test some samples, leave as-is if time is not that significant. The greater the distance (hops), the more likely the benefit of FTP.
0
 
LVL 1

Author Comment

by:davism
Comment Utility
I've been that FTP takes 1/2 the time less resource vs a copy.  

The sample that I provided doesn't necessary include that "originator" was you view it.  That is used as the directory specification but to remain consistent in usage we have the UNC name of the machine that is execution the function in that specification as well.  

The information of a sample that was provided to me was a 350 MB file transfered where a NET USE statement was defined (althought was not needed per the use of UNC's but I'm not sure if the person performing this sample understood that or not.)  That all being said, the 350 MB file took 8+ minutes to transfer over a LAN.  

The FTP of that same file was identified to take 4+ minutes to transfer.

Does this make any realistic sense?

Now, my understanding of FTP and the copy command is that they both use TCP/IP - one way or another.  

If that is truly the case, then what is the difference?  Authentication?  I would have to question that because the FTP process would still have to perform an NT authentication against the file that it's preparing to transmit as well unless you have anonymous FTP there still is an authentication role that has to be performed there as well.  

A copy command will still have to authenicate the file the same way.

Am I incorrect there?

If not, is that FTP taking 1/2 the amount of time a realistic assessment?

0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
> Now, my understanding of FTP and the copy command is that they both use TCP/IP - one way or another.
 
Not mine. Proof, is that your use of 'net' command, does not require tcp, netbios sufficient.

Take your sample mapping. but permit me to abbrev to j: and k:, let us say that these can be different shares on the same server.

But you are not at the server. You are at workstation. That is where you say 'copy'. The copy program is resident in your ram, not server ram. Permit another oversimplification, that to copy-file, you read sector in, write sector out, until done. So workstation pulls sector off of one place of server, then sends it back so it can be stored properly. For copying the file from one server location/share, to another, on the same server, there was really no need for the network traffic at all, yet the process required it to be moved on the wire twice, first to the desktop, then back to the server.

This is one of main drawbacks to the process of a 'simple' copy command. Twice the necessary work on the network, twice the bandwidth.

A complete comparison must consider topology, packet sizes, etc. With TCP/IP we have had some real programs, that took data and massaged it, then stored it, taking severl minutes, even up to 20 min. When gurus improved process by moving a server to gigabit access, the application(s) completed in 4 hours to overnight. Speed, such as networking, is not complete story. Packet sizes, overhead in tcp for negotiating sizes, readiness to receive, these can go from nearly insignificant to extremely significant especially when the types of routing devices are different and multiple protocols are used. FTP must go tcp/ip, while 'copy' does not. Thus there is no one single answer, and any answer that is true today, may not apply on another day after networking is upgraded. What about terminal server, terminal services?

General rule of thumb tho', is that the copy command is slower. And I think I've indicated main reason, introduction of third party / intermediary to a copying process.

> The FTP of that same file was identified to take 4+ minutes to transfer

This is best, IMO method to determine. Run some big files between the areas you really care about, and pursue it from there. Just beware of Maximimums for packet sizes, and try to maintain consistency throughout LAN/WAN intranet.

If glutton for info, take NT server, fire up its Network Monitor, filter on the devices used, then run copy and TCP commands. Look into the data area and see how much each method stuffs into packets. While I don't know, Generally, I'd guess that TCP does a better job of filling packets, while the copy command would have lower limit based upon sector size.
0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
> The FTP of that same file was identified to take 4+ minutes to transfer

actually, this looks like you answered your question earlier
0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
> If that is truly the case, then what is the difference?  Authentication?

Not NT world. MS is oft criticised for the opposite, authenticating a little too little up front, and not rechecking of authentication later when it really ought too. So I doubt any major difference there that would impact.

> A copy command will still have to authenicate the file the same way.

Only guessing, I tend to agree. Authentication for both would be more up front, for time consumptive tasks, and most of later traffic would be pre-agreed to as pieces of a predefined package.

You might want to look up alternatives, such as ncopy, for why they think they run better than the copy command. I'm sure it would make more sense by their authors than how I do here.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:davism
Comment Utility
Understand.  I apologize for the statement about the COPY command utilizing TCP/IP.  I forgot about the other protocols (i.e. NETBIOS, etc).  Probably because I've grown so accustomed to using TCP/IP.

However, you elude to another variable in that COPY situation being that of the "originator" (i.e. workstation) but that's not the case.  It's straight server to server with no intermediary. It would/is done directly from the server to the other server so I don't understand the "twice the necessary work...twice the bandwidth.".  Woud that situation apply in a server-to-server scenario?  On some occasions, it would be done through terminal services onto the "source" server copying to the "destination" and I'm not too sure how terminal services utilizes that RAM.  So, that could play a role.  But 9.9 times out of 10 it'll be from the server coping to the other server with no intermediary (i.e. workstation) involved.  

If that intermediary was involved.  Then, yes, definitely and without question would I agree that FTP was faster.  

I also agree with a true comparision taking into account the topology, network utilization (to include switches/routers, etc).  I posed that question to the individual who had done that sample that I spoke of previously.  That wasn't done and I believe as well that the sample WAS performed with an "intermediary" (i.e. workstation).  A true comparison/benchmark has to compare apples to apples with the only difference is that which is being compared.  

I still wonder/question...if FTP is so much faster as indicated in the sample that was provided to me - regardless of the conditions or the "apples to apples" comparison...if FTP is so much faster why wouldn't have MS have made the COPY command (or a "PASTE" in a Windows Explorer case) a flavor of FTP (hidden as "FTP" but executing a comparable functions as true "FTP")??  

Is that the million dollar question?

There must be some reason/benefit as to why that wasn't done.

Would you agree?  

This discussion is very enlightening and I very much enjoy this conversation.  I truly hope you are not taking offense or interpreting what I'm saying the wrong way.  I'm not a walking AD for MS, COPY or FTP.  Just trying to gather an understanding so that there is some reasoning to the "madness" (i.e. justification.) :-)  
0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
I am 3rd party, or at least am forgetful of things I may have known. I know not, for example, what may lie beneath a drag & drop from one window to another. For me it is now all amazingly fast compared to before, while the mappings involved can be rather strange.

But guessing.
Copy is old word of microsoft, never really revised well. They used to, for example have diskette copy progams, that would require swapping of one disk for another. Well, there was little ram for the buffer. But eventually, ram got bigger than diskette, ram got taught to swap out to harddrive, but still we had to swap diskettes, trying to remember which time was which one, source or destination. Late to upgrade.

Thusly, I suspect copy command itself is limited by an artificial sector count. Possibly only able to transfer a single sector at a time, or however that's defined.

FTP being also quite old, but my guess it that it fills the packet, for it has no sense of sizings for floppy, just for the wire. So define a packet, fill it up, send another, same size. Now how'd either one define sector? 256? 512? Either way, when packets can go to 1500, FTP has plenty of room to fit in multiple sectors compared to copy cmd.

For FTP there is a spec, among rfc's for packet content. One could look it up. But that's rules, and with MS windoze, the more accurate answer comes IMO not from books but from personal observation. That's where I brought up MS Network Monitor as the likeliest common ground for defining packet content of the two methods, as well as the usage of extra packets, to communicate, sync, or whatever, that may add delays.

Long ago I thought I'd heard that xCopy did better at packetization than copy. I asked and was told no, it is just as bad as copy command.  If so, then in your scenario it'd be same deal when comparing to FTP.

In comparison to HTML, rather, HTTP copy, I consider both FTP and Copy to be far superior and more reliable.

> There must be some reason/benefit as to why that wasn't done

Problem, big one, for both MS & Novell, was they learned to buy into unix code early on, but neither did much with using it, or incorporating into their environment. With MS - which stack is being used? (and then they abuse stack layers.. stability) With Novell, they came up with extra flavors of "nearly TCP/IP" confusing even their own developers. Try their latest free client and see if you can choose protocol to use.. intelligently. MS problem is no whizbang newly schooled programmer wanting to be seen near any code let alone old code, other than for flashy new wizards, and, to maintain popular base, had to retain parts that break as few existing programs as possible. We wanted each OS to copy to the other, so the new one has to talk old ways, for it all to happen.

Interesting side question: how has clipboard evolved? It has.

Possibly, it remains as 3rd party for drag|drop copy process. Example, you can use mouse to copy a file. Before you decide upon a destination, the OS has done some work. You can now use mouse to paste same file into each directory. But I think how it uses clipboards for such things (such as for sector buffer area) can vary from one OS to another.

Now in a lot of this, there are alternate tuning parameters, such as in registry. Overall, we are likely best with default, for overall performance.

One simple example had a remote control program work with novell ipx & win3.1 well, then upgraded to new version that behaved better for tcp & Win95.  Speed difference mainly in registry choices re: packetization parameters.

> why wouldn't have MS have made the COPY command

this command predates MS having any tcp capability at all
Unice commands, really ran different permission levels, and task priorities, compared to MS.

> It's straight server to server

Bear now in mind that, while MS sells nearly same SW for desktop as for server, they are tuned for their own needs. Desktop commands, for example, should be expected to run slower on a server than on a desktop, and vice versa. For same OS level.

Classes: FTP for server; Copy for desktop.  Use either commandset on other platform and one should expect lowered performance, based on platform tuning, all else equal.
0
 
LVL 1

Author Comment

by:davism
Comment Utility
Unfortunately, I don't know if I can agree with the MS COPY command was around before MS.  The CONCEPT of a COPY or flavor of a COPY was definitely around.  I can agree to that.  But the MS COPY command being derived from something else, no, I can't - in concept, however - definitely.  I also find it hard to imagine that with all the growth of MS over the years since MS-DOS and Windows 3.1 that MS left that copy command, static.

I've finally got around to just testing this myself.  

First, I started out with a test on my local machine.  One test dealing with an FTP from my local machine to the same local machine (just a different directory).  I did the same with a COPY statement.  I dealt with a 133 MB file.  Interestingly, the COPY command was 21 seconds FASTER than the FTP.  However, I didn't want to accept that fact either.  So...

I went on through Terminal Services to an the actual server that would be doing the majority of the work. I moved the test scenarios (.BAT) files over to that machine.  I picked a 75 MB zipped file to "work" with.  The destination server did in fact have FTP already installed so I used it.  Now, from the "source" machine, I executed the .BAT files to perform the tests.  Interestingly, again a 75 MB zip file.  The FTP was quicker...it took 9 seconds.  The COPY file took 12 seconds.  I set everything back again and I ran it a second time - just to be sure.  The same result.  Now, the 3 second difference, IMO, is so irrelavent isn't not even worth concern.  

I have to believe that the difference is so small that isn't not justified for concern.  Heck, even on my own machine the FTP was slower.  Not taking that for granted, I did other tests on the actual network and actual machines I was dealing with.  The results showed the opposite.  But difference were so small in ALL conditions that is doesn't, IMO, justify any concern.  At least, in a W2K, NT 4.0 type setting.  Unix/Linux (not too sure if it's a good idea to lump those together. :-) ) using Samba would introduce another aspect but again IMO, it's not justified for major concern as we don't perform A LOT of activity through Samba on those Unix or Linux shares.  Maybe .5 percent.

How would you characterize those test findings?  Would you agree with not justified enough for concern or is there possibily something that you believe I may have neglected?

I do know that network utilization was not verified at the time.  But all-in-all, the test simulated what was provided to me, previously, as a sample.  I'm not seeing nearly a 100% performance improvement as what was indicated to me.  Do you have any ideas on what may have attributed to that?  (All I can think of is that test/sample being done in that previous sample was with the use of an intermediary (i.e. workstation.))  My test wasn't even close to a 100% improvement in favor of FTP.  Can you think of anything else that may attribute to those results of the 100% performance improvement from that previous sample which was provided to me?

0
 
LVL 44

Expert Comment

by:CrazyOne
Comment Utility
I thought I would confuse the issue a little more. As far as Drag/Drop is concerned it doesn't use the COPY command but calls on a function located in the Shell32.dll called SHFileOperation. It is probably faster then COPY or FTP but it has one caveat in that if it comes across a file that causes a sharing violation then it won't continue to completion of its operation. In other words it just stops doing what it is doing and won't resume. Also XCOPY was revamped a little when Win2000 was released. It actually added switches that are only usable for NT based OS's but I cannot attest to its speed in relationship to FTP or COPY.

Anyway davism as far as which is faster would seem to me that it is highly dependent on the environment overall. In some instances FTP may be faster but in other instances COPY maybe faster and it retains the original file attributes and date and time stamp as Sun mentioned. In other words if the environment isn't a constant like network traffic, machine speed and capabilities, hub and router and firewall variances, etc then I don't think the question which is faster can actually be answered with any true amount of accuracy. I would suggest sticking with what your testing indicates but to retest from time to time especially if the environment has changed.



Form the Windows SDK

SHFileOperation

Performs a copy, move, rename, or delete operation on a file system object.

WINSHELLAPI int WINAPI SHFileOperation(

    LPSHFILEOPSTRUCT lpFileOp      
   );      
 

Parameters

lpFileOp

Pointer to an SHFILEOPSTRUCT structure that contains information the function needs to carry out the operation.

 

Return Values

Returns zero if successful or nonzero if an error occurs.

See Also

SHFILEOPSTRUCT




SHFILEOPSTRUCT

Contains information that the SHFileOperation function uses to perform file operations.

typedef struct _SHFILEOPSTRUCT { // shfos  
    HWND         hwnd;
    UINT         wFunc;
    LPCSTR       pFrom;
    LPCSTR       pTo;
    FILEOP_FLAGS fFlags;
    BOOL         fAnyOperationsAborted;
    LPVOID       hNameMappings;
    LPCSTR       lpszProgressTitle;
} SHFILEOPSTRUCT, FAR *LPSHFILEOPSTRUCT;
 

Members

hwnd

Handle of the dialog box to use to display information about the status of the operation.

wFunc

Operation to perform. This member can be one of the following values:

FO_COPY      Copies the files specified by pFrom to the location specified by pTo.
FO_DELETE      Deletes the files specified by pFrom (pTo is ignored).
FO_MOVE      Moves the files specified by pFrom to the location specified by pTo.
FO_RENAME      Renames the files specified by pFrom.
 

pFrom

Pointer to a buffer that specifies one or more source file names. Multiple names must be null-separated. The list of names must be double null-terminated.

pTo

Pointer to a buffer that contains the name of the destination file or directory. The buffer can contain mutiple destination file names if the fFlags member specifies FOF_MULTIDESTFILES. Multiple names must be null-separated. The list of names must be double null-terminated.

fFlags

Flags that control the file operation. This member can be a combination of the following values:

FOF_ALLOWUNDO      Preserves undo information, if possible.
FOF_CONFIRMMOUSE      Not implemented.
FOF_FILESONLY      Performs the operation only on files if a wildcard filename (*.*) is specified.
FOF_MULTIDESTFILES      Indicates that the pTo member specifies multiple destination files (one for each source file) rather than one directory where all source files are to be deposited.
FOF_NOCONFIRMATION      Responds with "yes to all" for any dialog box that is displayed.
FOF_NOCONFIRMMKDIR      Does not confirm the creation of a new directory if the operation requires one to be created.
FOF_RENAMEONCOLLISION      Gives the file being operated on a new name (such as "Copy #1 of...") in a move, copy, or rename operation if a file of the target name already exists.
FOF_SILENT      Does not display a progress dialog box.
FOF_SIMPLEPROGRESS      Displays a progress dialog box, but does not show the filenames.
FOF_WANTMAPPINGHANDLE      Fills in the hNameMappings member. The handle must be freed by using the SHFreeNameMappings function.
 

fAnyOperationsAborted

Value that receives TRUE if the user aborted any file operations before they were completed or FALSE otherwise.

hNameMappings

Handle of a filename mapping object that contains an array of SHNAMEMAPPING structures. Each structure contains the old and new path names for each file that was moved, copied, or renamed. This member is used only if fFlags includes FOF_WANTMAPPINGHANDLE.

lpszProgressTitle

Pointer to a string to use as the title for a progress dialog box. This member is used only if fFlags includes FOF_SIMPLEPROGRESS.

 

Remarks

If pFrom or pTo are unqualified names, the current directories are taken from the global current drive and directory settings as managed by the GetCurrentDirectory and SetCurrentDirectory function



The Crazy One
0
 
LVL 24

Accepted Solution

by:
SunBow earned 50 total points
Comment Utility
> since MS-DOS and Windows 3.1 that MS left that copy command, static.

Dos prior to v5 did very poor for any networking, Win3.1 not much better. Adding networking capability to OS is independent of packaging & buffering.

> COPY command was 21 seconds FASTER than the FTP

From same machine to itself, networking can be skimped on

> I went on through Terminal Services

Like Windoze, this now introduces overhead to the desktop, for progress indicators. We all know Windoze is slow. A fair representation 'could be' if you also used an ftp for windows program that also had similar periodic user feedback during the transmission.

> Now, the 3 second difference, IMO, is so irrelavent isn't not even worth concern.  

Now you have the answers. For occasional transfer, three sec on its own, while some may care, you do not, so let it go. (except perhaps as a personal test for "network health" - it should be relatively consistent over time)

Consider a responsibility for deploying a package to 100,000 users. Now that 3 sec can add up quick. Make it a daily update package and it is verr important.

But on a small scale, there are so many other factors like training, security, packaging, authentication, development(time) etc, that speed difference can be of less concern to many.

> As far as Drag/Drop is concerned it doesn't use the COPY command but calls on a function
> capabilities, hub and router and firewall variances,

My wonderment, is that for smaller files I used to get to see some in-progress displays, but over time and upgrades, I got to drag really big files, greater distances, comparatively, and not get to see the darn moving folder, or whatever. How much was M$ OS, and how much changing networking, adding switches, I dunno. What I know it wasn't was NIC. Same old 10/100 NIC, running at 10, when I now have upgrade choice, it is less concern. For xmits < 1sec now, a ten-fold improvement would not be so noticeable.

Apples to Oranges.

If you have to toss in others ti discourse (Samba), and are glutton, try on Unice's:

similar copy commands, cp and uucp. (latter is unix-to-unix prefix).

IMO, tuning like that is best done in real world. It is like difference between mathematics (concept) and physics (reality), or like MS' books list answers for tests, not for real-world behavior of their products. Just don't tell anyone, most like the kinds of easy answers they can look up in Cliff-Notes (ha-ha).

We haven't even begun to mention the kinds of filters (virus|worm) that can slow down a transmission. Or caches that can speed up (crossed fingers) or slow down xMit. Or the BulkTransfer tools.
0
 
LVL 1

Author Comment

by:davism
Comment Utility
Sorry for the delay in responding.  

Thanks for the information and the enlightened discussion.  As you indicated, I answered my own question.  I even spoke to the individual who provided that sample and the sample wasn't performed as I thought.  It neglected some of the simple tasks being performed.

>Consider a responsibility for deploying a package to 100,000 users. Now that 3 sec can add up quick.
Make it a daily update package and it is verr important.

I completely agree, however, we don't process at that volume.  That is my fault for not mentioning.  Data content wise we're processing millions and millions of records but those are all contained within the file.  (1 FTP obtaining a file and that file has hundreds, thousands, hundreds of thousands or millions.)  It just depends.  But it's 1 file being transmitted.  Now, we might have to transmit the 1 file situation 50 times but again, it's nowhere near 100,000.  :-)  

Again, thanks for the enlightening discussion.  I enjoyed it very much and I appreciate it.  And again, I feel and I believe you were correct...I answered my own question.  :-)

Thanks
0
 
LVL 24

Expert Comment

by:SunBow
Comment Utility
Many Happy Returns..........   :o)
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Introduction How to create multiboot configuration with XP\Vista and Windows 7 on it? And most important question - how to do this correctly so not to have any kind of nightmares we get when system gets screwed? First of all one should realize t…
Windows 10 is here and for most admins this means frustration and challenges getting that first working Windows 10 image. As in my previous sysprep articles, I've put together a simple help guide to get you through this process. The aim is to achiev…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now