Help with a script to copy the contents of multiple folders to similarly named folders on another drive.

I have around 100 folders. I am moving them into sharepoint online. Due to restrictions in sharepoint I have ended up creating a seperate sharepoint library for each folder.

I have used powershell to create identically named libraries in sharepoint as th folders I already have locally.

I have mapped a network drive to the sharepoint site and now have two windows open on my PC.

One showing the folders on the local drive and another showing the libraries in sharepoint - which appear just as folders too.

I have been manually going into each folder and copying the contents over which works fine but I would like to run some kind of script to automate this process. I have a text file containing the list of folder names if that helps.

Any advice?
LVL 1
roy_battyDirectorAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jason CrawfordTransport NinjaCommented:
I'm not sure what kind of restriction you're referring to, but I'll help write the script.  I'll need the source and target folder structure.
NVITEnd-user supportCommented:
I am not familiar with Sharepoint and IDK if there are differences but... since you already have drive letters, have you tried XCOPY or ROBOCOPY in a CMD prompt? e.g.
xcopy "y:\folder" "z:\folder\" /e /y /i

Open in new window

or...
robocopy "y:\folder" "z:\folder\" /e

Open in new window


With the text file...
Note: Remove the ECHO to run for real.
for /f "tokens=*" %a in (textfile.txt) do ECHO xcopy "y:\%a" "z:\%a\" /e /y /i

Open in new window

David Johnson, CD, MVPOwnerCommented:
There are a million and one ways to do this and if you gave an example of the source and destination directories it would be helpful

i.e. create a csv
source,destination
x:\mysource\a,z:\abc\a
\\server99\share100\b,z:\abc\b

Open in new window

$paths = import-csv csvname.csv
foreach($path in $paths){ Copy-Item -Path $path.source -Destination $path.destination –Recurse }

Open in new window

or a oneliner
import-csv csvname.csv | % { copy-item $_.source $_destination -recurse }

Open in new window

Acronis True Image 2019 just released!

Create a reliable backup. Make sure you always have dependable copies of your data so you can restore your entire system or individual files.

roy_battyDirectorAuthor Commented:
OK I have a file called "folderlist.csv" which when opened in excel has one column of data. A header row containing "Library Name" and then a list of 100 library names below it. This is the file I used to create the libraries in Sharepoint.

In the t:\ drive is all the folders which match the libraries in sharepoint and is the source of all the files.

The s:\ drive is connected directly to the sharepoint site, so I see the same folder structure in there. This is the destination.

Does that help?
NVITEnd-user supportCommented:
Adjusting my original post...
- Make a .bat file of this code.
- Set the right side of CopyFolders to the folder that folderlist.csv resides. I used c:\utils
- Open a CMD windows to run the .bat
- This code is in debug mode, which just echos the command that would run. Remove the ECHO to run for real
- In your folderlist.csv, you should put just 1 or 2 lines to test it. When satisfied, remove the ECHO. Then run the .bat again. This time, it should copy for real. If that works, remove the folders just copied from the .csv. Then add the remaining folders to the .csv. Then do the final run.

@echo off
set CopyFolders=c:\utils\folderlist.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do ECHO xcopy "t:\%%a" "s:\%%a\" /e /y /i

Open in new window


If you encounter errors in the CMD window during the run, please post exact text here.
roy_battyDirectorAuthor Commented:
OK NVIT I have done as you suggested and get the following errors. I presume it is trying to create the folders on the destination drive. These folders have already been created so I presume this is why I get this error.

How do I amend the batch file to take into account the folder structure is already in place. Note that the top level folder does contain sub folders along with files.

C:\Users\Lisa\Desktop>copy.bat
File creation error - The directory is not empty.

Unable to create directory - V:\Letters
0 File(s) copied
File creation error - The directory is not empty.

Unable to create directory - V:\linked in
0 File(s) copied
File creation error - The directory is not empty.

Unable to create directory - V:\Literature
0 File(s) copied

Open in new window

NVITEnd-user supportCommented:
Please post your code.

Did you run it with the ECHO xcopy as a test? If so, did that show the folder names correctly?

For now, remove the top @echo off line. You can add it back later
Run again and post text here.
roy_battyDirectorAuthor Commented:
My code (including xcopy, the output of which I posted in my previous post) is :

@echo off
set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "t:\%%a" "v:\%%a\" /e /y /i

Open in new window


The folders listed in the output looks correct to me.

Without the @echo off the output is:
C:\Users\Lisa\Desktop>copy.bat

C:\Users\Lisa\Desktop>set CopyFolders=c:\ed\folderlist2.csv

C:\Users\Lisa\Desktop>for /F "tokens=* skip=1" %a in (c:\ed\folderlist2.csv) do
xcopy "t:\%a" "v:\%a\" /e /y /i

C:\Users\Lisa\Desktop>xcopy "t:\Letters" "v:\Letters\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\Letters
0 File(s) copied

C:\Users\Lisa\Desktop>xcopy "t:\linked in" "v:\linked in\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\linked in
0 File(s) copied

C:\Users\Lisa\Desktop>xcopy "t:\Literature" "v:\Literature\" /e /y /i
File creation error - The directory is not empty.

Unable to create directory - V:\Literature
0 File(s) copied

Open in new window

roy_battyDirectorAuthor Commented:
Any more feedback?
David Johnson, CD, MVPOwnerCommented:
remove the /i parameter
roy_battyDirectorAuthor Commented:
OK Thanks for the advice so far.

I have been getting an "invalid drive specification". After some research I have discovered that xcopy doesnt like drive letters.

I have therefore amended the script as below:

set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "\\localhost\shared\%%a" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a" /e /y /i

Open in new window


I am now getting an error that it cannot create the directories in the destination. I imagine that the script is trying to create folders listed in my .csv file before copying over any sub-folder/files.

As the destination already contains the folders ( they are actually sharepoint libraries) I want my script to copy the contents of the folders over without trying to recreate each top level folder first.

How do I achieve this?
roy_battyDirectorAuthor Commented:
OK I sussed it. Just add a  \* to the end of each path.

So here's what worked for me:

set CopyFolders=c:\ed\folderlist2.csv
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do xcopy "\\localhost\shared\%%a\*" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a\*" /e /y /i

Open in new window


The final question is how can I create a log file of the whole process so I can check for errors after it has run?
NVITEnd-user supportCommented:
> ...how can I create a log file of the whole process
Try this
set CopyFolders=c:\ed\folderlist2.csv
set FNLog=c:\ed\copylog.txt
for /f "tokens=* skip=1" %%a in (%CopyFolders%) do (
  xcopy "\\localhost\shared\%%a\*" "\\companyname.sharepoint.com@SSL\DavWWWRoot\%%a\*" /e /y /i
  if %errorlevel% neq 0 >> "%FNLog%" echo ERROR %errorlevel% during xcopy "%%a"
)

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
roy_battyDirectorAuthor Commented:
OK I will give that a try. Also currently the script stops when it encounters an error. How do i make it skip any errors whilst adding entries to the log?
NVITEnd-user supportCommented:
It should continue during errors. Still, when it errors, please post it.
roy_battyDirectorAuthor Commented:
Yes my mistake it does skip errors. I havent had any yet but it tells me there were no errors after copying each folder.

I am now running it against the whole list of folders and it seems to be working really well.

I will report back with the outcome.
NVITEnd-user supportCommented:
OK. Any errors should log to file c:\ed\copylog.txt
roy_battyDirectorAuthor Commented:
Great. That works perfectly.

Thanks for the help.
NVITEnd-user supportCommented:
I'm glad for the chance. Have a nice day.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.