Help with a script to copy the contents of multiple folders to similarly named folders on another drive.

roy_batty
roy_batty used Ask the Experts™
on
I have around 100 folders. I am moving them into sharepoint online. Due to restrictions in sharepoint I have ended up creating a seperate sharepoint library for each folder.

I have used powershell to create identically named libraries in sharepoint as th folders I already have locally.

I have mapped a network drive to the sharepoint site and now have two windows open on my PC.

One showing the folders on the local drive and another showing the libraries in sharepoint - which appear just as folders too.

I have been manually going into each folder and copying the contents over which works fine but I would like to run some kind of script to automate this process. I have a text file containing the list of folder names if that helps.

Any advice?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Commented:
I'm not sure what kind of restriction you're referring to, but I'll help write the script.  I'll need the source and target folder structure.
NVITEnd-user support

Commented:
I am not familiar with Sharepoint and IDK if there are differences but... since you already have drive letters, have you tried XCOPY or ROBOCOPY in a CMD prompt? e.g.
xcopy "y:\folder" "z:\folder\" /e /y /i

Open in new window

or...
robocopy "y:\folder" "z:\folder\" /e

Open in new window


With the text file...
Note: Remove the ECHO to run for real.
for /f "tokens=*" %a in (textfile.txt) do ECHO xcopy "y:\%a" "z:\%a\" /e /y /i

Open in new window

Top Expert 2016

Commented:
There are a million and one ways to do this and if you gave an example of the source and destination directories it would be helpful

i.e. create a csv
source,destination
x:\mysource\a,z:\abc\a
\\server99\share100\b,z:\abc\b

Open in new window

$paths = import-csv csvname.csv
foreach($path in $paths){ Copy-Item -Path $path.source -Destination $path.destination –Recurse }

Open in new window

or a oneliner
import-csv csvname.csv | % { copy-item $_.source $_destination -recurse }

Open in new window

OWASP: Avoiding Hacker Tricks

Learn to build secure applications from the mindset of the hacker and avoid being exploited.

roy_battyDirector

Author

Commented:
OK I have a file called "folderlist.csv" which when opened in excel has one column of data. A header row containing "Library Name" and then a list of 100 library names below it. This is the file I used to create the libraries in Sharepoint.

In the t:\ drive is all the folders which match the libraries in sharepoint and is the source of all the files.

The s:\ drive is connected directly to the sharepoint site, so I see the same folder structure in there. This is the destination.

Does that help?
NVITEnd-user support

Commented:
Adjusting my original post...
- Make a .bat file of this code.
- Set the right side of CopyFolders to the folder that folderlist.csv resides. I used c:\utils
- Open a CMD windows to run the .bat
- This code is in debug mode, which just echos the command that would run. Remove the ECHO to run for real
- In your folderlist.csv, you should put just 1 or 2 lines to test it. When satisfied, remove the ECHO. Then run the .bat again. This time, it should copy for real. If that works, remove the folders just copied from the .csv. Then add the remaining folders to the .csv. Then do the final run.

@echo off
set CopyFolders=c:\utils\folderlist.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do ECHO xcopy "t:\%%a" "s:\%%a\" /e /y /i

Open in new window


If you encounter errors in the CMD window during the run, please post exact text here.
roy_battyDirector

Author

Commented:
OK NVIT I have done as you suggested and get the following errors. I presume it is trying to create the folders on the destination drive. These folders have already been created so I presume this is why I get this error.

How do I amend the batch file to take into account the folder structure is already in place. Note that the top level folder does contain sub folders along with files.

C:\Users\Lisa\Desktop>copy.bat
File creation error - The directory is not empty.

Unable to create directory - V:\Letters
0 File(s) copied
File creation error - The directory is not empty.

Unable to create directory - V:\linked in
0 File(s) copied
File creation error - The directory is not empty.

Unable to create directory - V:\Literature
0 File(s) copied

Open in new window

NVITEnd-user support

Commented:
Please post your code.

Did you run it with the ECHO xcopy as a test? If so, did that show the folder names correctly?

For now, remove the top @echo off line. You can add it back later
Run again and post text here.
roy_battyDirector

Author

Commented:
My code (including xcopy, the output of which I posted in my previous post) is :

@echo off
set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "t:\%%a" "v:\%%a\" /e /y /i

Open in new window


The folders listed in the output looks correct to me.

Without the @echo off the output is:
C:\Users\Lisa\Desktop>copy.bat

C:\Users\Lisa\Desktop>set CopyFolders=c:\ed\folderlist2.csv

C:\Users\Lisa\Desktop>for /F "tokens=* skip=1" %a in (c:\ed\folderlist2.csv) do
xcopy "t:\%a" "v:\%a\" /e /y /i

C:\Users\Lisa\Desktop>xcopy "t:\Letters" "v:\Letters\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\Letters
0 File(s) copied

C:\Users\Lisa\Desktop>xcopy "t:\linked in" "v:\linked in\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\linked in
0 File(s) copied

C:\Users\Lisa\Desktop>xcopy "t:\Literature" "v:\Literature\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\Literature
0 File(s) copied

Open in new window

roy_battyDirector

Author

Commented:
Any more feedback?
Top Expert 2016

Commented:
remove the /i parameter
roy_battyDirector

Author

Commented:
OK Thanks for the advice so far.

I have been getting an "invalid drive specification". After some research I have discovered that xcopy doesnt like drive letters.

I have therefore amended the script as below:

set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "\\localhost\shared\%%a" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a" /e /y /i

Open in new window


I am now getting an error that it cannot create the directories in the destination. I imagine that the script is trying to create folders listed in my .csv file before copying over any sub-folder/files.

As the destination already contains the folders ( they are actually sharepoint libraries) I want my script to copy the contents of the folders over without trying to recreate each top level folder first.

How do I achieve this?
roy_battyDirector

Author

Commented:
OK I sussed it. Just add a  \* to the end of each path.

So here's what worked for me:

set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "\\localhost\shared\%%a\*" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a\*" /e /y /i

Open in new window


The final question is how can I create a log file of the whole process so I can check for errors after it has run?
End-user support
Commented:
> ...how can I create a log file of the whole process
Try this
set CopyFolders=c:\ed\folderlist2.csv
set FNLog=c:\ed\copylog.txt
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do (
  xcopy "\\localhost\shared\%%a\*" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a\*" /e /y /i
  if %errorlevel% neq 0 >> "%FNLog%" echo ERROR %errorlevel% during xcopy "%%a"
)

Open in new window

roy_battyDirector

Author

Commented:
OK I will give that a try. Also currently the script stops when it encounters an error. How do i make it skip any errors whilst adding entries to the log?
NVITEnd-user support

Commented:
It should continue during errors. Still, when it errors, please post it.
roy_battyDirector

Author

Commented:
Yes my mistake it does skip errors. I havent had any yet but it tells me there were no errors after copying each folder.

I am now running it against the whole list of folders and it seems to be working really well.

I will report back with the outcome.
NVITEnd-user support

Commented:
OK. Any errors should log to file c:\ed\copylog.txt
roy_battyDirector

Author

Commented:
Great. That works perfectly.

Thanks for the help.
NVITEnd-user support

Commented:
I'm glad for the chance. Have a nice day.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial