A faster way to create sub folders

Hello experts. I have been running the following script which a fellow experts created for me here some time ago. The script works perfectly except it is taking hours to run due to the number of folders that need to be checked and created. I am wondering if there is a more efficient way to check for the existence of folders and creates folders that are missing (powershell possibly). I have reduced the number of folders in the script to keep my post concise but they are significantly more folders checked & created. The script is saved and run as a .cmd

Script

cd /d D:\Data\
setloca
for /f "tokens=* delims=" %%A in ('dir D:\Data\*.* /b /ad') do call :makedirs "%%A"
goto :eof

:makedirs
set rootdir=%1
mkdir %rootdir%\Corro"
mkdir %rootdir%\Corro\2003
mkdir %rootdir%\Email
mkdir %rootdir%\Excel
mkdir %rootdir%\Statutory
mkdir %rootdir%\MYOB
mkdir %rootdir%\MYOB\2008

:end
Amiga-2000Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

becraigCommented:
Here is a quick option:
There are two inputs
1. a text file with all your directories
2. a text file with all your servers.



gc dirs.txt | % {$dirs +=
@"
mkdir $_

"@
}
Get-Content "ServerList.txt" | %{
  # Execute the jobs in parallel
  Start-Job -Scriptblock {Param($dirs); $dirs | % {mkdir $_} } -ArgumentList $dirs
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
  Start-Sleep 10
}
# Getting the information back from the jobs
Get-Job | Receive-Job

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Amiga-2000Author Commented:
Hi, thanks for response. I'm afraid I don't really understand it though. What are the two input files and what do I enter in them?

Thanks
0
becraigCommented:
So you say you have a LIST of directories to create ?
So if the directories are all to be created on each computer we are just populating this as a variable from a text file:
textfile1:
%rootdir%\Corro
%rootdir%\Corro\2003
%rootdir%\Email
%rootdir%\Excel
%rootdir%\Statutory
%rootdir%\MYOB
%rootdir%\MYOB\2008


Textfile 2 will be your list of servers:
server1
server2
server3 etc    or you can use whatever source you use for feeding this in.

Is this a part of a larger script ?
0
Introducing the "443 Security Simplified" Podcast

This new podcast puts you inside the minds of leading white-hat hackers and security researchers. Hosts Marc Laliberte and Corey Nachreiner turn complex security concepts into easily understood and actionable insights on the latest cyber security headlines and trends.

footechCommented:
Can't say I know the answer, especially whether PS would be any faster than your batch file for this.  It'd be interesting to see if parallel processing could help.  One idea though, your list of folders to be created could be trimmed such that if you have
   mkdir %rootdir%\Corro
   mkdir %rootdir%\Corro\2003
you only need to include the longest path
   mkdir %rootdir%\Corro\2003
and it will create all the necessary folders.
0
becraigCommented:
Valid input Foo, my thinking here is parallel processing should show gains over serial processing. I might be wrong but it seems only logical.

Your suggestion on the longest path is quite valid I did not really suggest it as I only saw one occurrence, if there are multiple entries such as this I would suggest Amiga employ that solution as well.

I also can't say I know the "answer" however if the concern is the amount of time it takes to complete, as I said above I can logically see parallel processing as a more efficient way than serial.
0
J-LCommented:
Here's some more condense code, but I'm not able to test it here.

$Folders = Get-Content .\FoldersToCreate.txt
$ServerList = Get-Content .\Servers.txt
Invoke-Command -ComputerName $ServerList -ScriptBlock {
    param ($folders)
    foreach ($dir in (Get-ChildItem d:\data | ? {$_.PSIsContainer})) {
        $null = New-Item -Path ( $folders | % { Join-Path $dir $_ } ) -ItemType Directory -ea 0
    }
} -ArgumentList (,$Folders)

Open in new window


Before targeting all servers (which I assume here are in the servers.txt, one per line), you read out a list of folders to create (also one per line, just the name of the folder).

servers.txt
computer1
computer2
computer3

Open in new window

folders.txt
Corro
Corro\2003
Email
Excel
Statutory
MYOB
MYOB\2008

Open in new window


Through some nifty little trick, using (,$Folders) as syntax, the array of folders is passed on to Invoke-Command, and on each server locally, the folders are then created.

If you just need to run it locally, use this shorter version:

$Folders = Get-Content .\FoldersToCreate.txt
foreach ($dir in (Get-ChildItem d:\data | ? {$_.PSIsContainer})) {
    $null = New-Item -Path ( $folders | % { Join-Path $dir $_ } ) -ItemType Directory -ea 0
}

Open in new window

0
Amiga-2000Author Commented:
Thanks all so far. I'm trying J-L local script

I created createfolders.ps1 root of d:

$Folders = Get-Content .\FoldersToCreate.txt
foreach ($dir in (Get-ChildItem d:\data | ? {$_.PSIsContainer})) {
    $null = New-Item -Path ( $folders | % { Join-Path $dir $_ } ) -ItemType Directory -ea 0
}

also in root of D: FoldersToCreate.txt with 5 sample folders to create

Open powershell as admin and run, no errors are thrown but no folders are made. How do I work out whats gone wrong :-)

Thanks
0
Amiga-2000Author Commented:
Arh - ok I see what it's doing

It's getting the folders from d:\data and recreating them in the root of D: with the required subfolders. That's almost what I need but I need the subfolders created in the existing d:\data path. Can it be changed to do that please?

Thanks
0
footechCommented:
Do you only have to create all these folders on a single machine or many?  If many, the process could certainly benefit from all the servers creating the folders concurrently.

Here's about the least for just the local machine (just modify the first two lines to suit you).
$folderList = "c:\temp\folderlist.txt"
$startFolder = "d:\data"
Set-Location $startFolder
New-Item (Get-Content $folderList) -ItemType directory | Out-Null

Open in new window

0
Amiga-2000Author Commented:
Hi

THanks, I'm not sure what to do with the code though (I've changed top two lines) but do I add this at start of the code. Thanks

ps yes I only need this to run on a single server.

Thanks again
0
footechCommented:
Wait a minute.  Went back and reread the code in your original question.  What I posted would not do the same thing.
I don't see any problem with J-L's code.  It looks like it exactly fits your requirements.  BTW, you can add the -whatif parameter to the New-Item command so that it will show what it would do, without actually creating any of the folders.
There has to be some folders already in d:\data, as the subfolders from the file are created in those folders.  This is the same way that the batch file worked.
0
J-LCommented:
I'm not sure it's the case, but you might need to add .Fullname in your Join-Path:
$folders | % { Join-Path $dir.FullName $_ } 

Open in new window


Since the -Path parameter of Join-Path expects a string value, maybe giving it the path object instead of the path string causes this.
0
Amiga-2000Author Commented:
Thanks, the last comment sorted out the path for me. Thanks everyone
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.