Convert old Scipt to Powershell

We have the following script that was written several years ago.  The purpose of this script is to change the permissions of all .JPG files within a directory and then change the file attribute to "A" so that it doesn't need to process these JPG's again.  However, because it's running on such a large amount of data, it takes over a week to run when all JPG's need to be processed.  I need to find a faster way to accomplish this task, but need assistance as I'm not an expert at scripting.  I thought converting this script to run in powershell would be ideal.  Can anyone assist me with this by re-writing this script in powershell?

@echo on
for /F "delims=*" %%F in ('xcopy /S /L "E:\DATA\*.jpg" \ ^| findstr /v File^(s^)') do (
  xcacls "%%F" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
  attrib -a "%%F"
)

Open in new window

ITPro44Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

QlemoBatchelor, Developer and EE Topic AdvisorCommented:
I doubt you can speed up the overall process.
Instead of XCopy and findstr, I would use dir /a:-a /s /b E:\DATA\*.jpg - but don't think that is faster.

In PowerShell the only pro is that the files get processed as soon as they are seen in the directory. The batch file above has first to process the complete directory, and then applies changes. This is not faster, but allows the process to get interrupted at any point, and restarting should be reasonable fast.
0
arnoldCommented:
Any reason why the settings is not applied once to the e:\Data\ directory or is the issue that files end up without inherit rights?

http://blogs.technet.com/b/heyscriptingguy/archive/2014/02/03/list-files-in-folders-and-subfolders-with-powershell.aspx
0
zalazarCommented:
I agree with Qlemo as I also think that converting it to the same code with PowerShell won't help you that much. This since the script is processing each file one by one which is slowing down the process very much. Maybe another tool then xcacls can update the permissions a bit faster but the better optimization would be to rewrite the script in a way that multiple threads of xcacls will be started at the same time.
0
Introducing the "443 Security Simplified" Podcast

This new podcast puts you inside the minds of leading white-hat hackers and security researchers. Hosts Marc Laliberte and Corey Nachreiner turn complex security concepts into easily understood and actionable insights on the latest cyber security headlines and trends.

ITPro44Author Commented:
I know robocopy can be multithreaded by using the /MT switch.  Could robocopy be utilized for this?
0
zalazarCommented:
Yes that's correct. Unfortunately robocopy can only copy NTFS permissions of files and dirs but it cannot change NTFS permissions of an existing file so you can't use it for this task.
I can try to look into a faster way of doing this task but it will cost some time to sort out.
0
arnoldCommented:
you could but xcopy and robocopy using the /L flag are only there to enumerate the files in the source matching a criteria.

What is the issue that prompted the need for setting these rights individually on the file, that setting on the entire directory will not work?
A better option since you are concerned with images, is to separate the images ...

Context for this setup may explain, i.e. you would like to grant additional rights to a group of people but only to a subset of files.

Note that this script keeps updating the same files everytime it runs.
using powershell, you would/should narrow the searc criteria to include recently created/modified files

a unix utility find will do the search/listing including

powershell, fileobject, and foreach you can then set the permissions within without the need to call xcacls ......
0
ITPro44Author Commented:
Unfortunately, we can't separate out the JPG's from the other files and this is why we can't set permissions on the top level folder.
0
ITPro44Author Commented:
My reason I mentioned robocopy is that I thought it would speed up the 1st stage of the process which is to gather a list of all file names that need to be processed.  Will it not do that?
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
No.
0
ITPro44Author Commented:
Qlemo - can you provide me with the script that would use "dir /a:-a /s /b E:\DATA\*.jpg" instead of xcopy and fidstr, as you suggested?  I'll give this a try.
0
ITPro44Author Commented:
zalazar - please do let me know if your able to find an alternative way that would make this faster.
0
zalazarCommented:
Sure, I will let you know.
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
@echo on
for /F "delims=*" %%F in ('dir /a:a /s /b E:\DATA\*.jpg') do (
  xcacls "%%F" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
  attrib -a "%%F"
)

Open in new window

The dir I provided originally was wrong. it listed all files having the archive attribute NOT set.

The PowerShell way is something like this, in case you want to test:
Get-ChildItem E:\Data -include *.jpg |
  ? { $_.Attributes -band [System.IO.FileAttributes]::Archive } |
  % {
    xcacls "$_.FullName" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
    $_.Attributes = $_.Attributes -bxor [System.IO.FileAttributes]::Archive
  }

Open in new window

As you can see, it is not really different, processing is very similar. The full .NET method to replace xcacls is more complex, and would not help to change speed either, so I just left the command.
0
ITPro44Author Commented:
Thanks Qlemo - for the first script, when I run it, get the following error.  

xcacls "E:\DATA\IMG_0001.JPG" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
 attrib -a "E:\DATA\IMG_0001.JPG"
)
ERROR: No mapping between account names and security IDs was done.

Open in new window


For the second powershell script, I copied it into a text file and gave it a PS1 extension name.  I then tried to run it in powershell by cd'ing into the directory of the file and then calling by using ".\text4.ps1"  I immediately get the following result.  Nothing was run or changed on the data :

PS C:\IT\SCRIPTS> ".\test3.ps1"
.\test3.ps1
PS C:\IT\SCRIPTS>

Open in new window


What do you suggest?
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Try the listed xcacls line as shown in the error message again (outside of the batch). It should report the same error, which tells that there is a unknown group or user provided.

The PS1 file did not run because of the double quotes - that results in a string. Remove the double quotes, or use the "exec" operator &:
.\test.ps1
& ".\test.ps1"
& '.\test.ps1'

Open in new window

0
ITPro44Author Commented:
I removed the double quotes and it still doesn't run.  I get this result instead.

PS C:\IT\SCRIPTS> .\test3.ps1
PS C:\IT\SCRIPTS>

Open in new window


My mistake on the xcacls error - that has been resolved.  thank you.
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Oops, forget to iterate thru subfolders. Shame on me.
Get-ChildItem E:\Data -include *.jpg -recurse |
  ? { $_.Attributes -band [System.IO.FileAttributes]::Archive } |
  % {
    xcacls "$_.FullName" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
    $_.Attributes = $_.Attributes -bxor [System.IO.FileAttributes]::Archive
  }

Open in new window

0
ITPro44Author Commented:
I now get another error using the update script.  The script changes the archive attribute from "A" to "N" but does not change any permissions on the files:

PS C:\IT\SCRIPTS> .\test3.ps1
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
ERROR: The system cannot find the file specified.
PS C:\IT\SCRIPTS>

Open in new window

0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Stupid me, seems not be my week (or month or year?).
Get-ChildItem E:\Data -include *.jpg -recurse |
  ? { $_.Attributes -band [System.IO.FileAttributes]::Archive } |
  % {
    xcacls "$($_.FullName)" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
    $_.Attributes = $_.Attributes -bxor [System.IO.FileAttributes]::Archive
  }

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ITPro44Author Commented:
It now changes the archive attribute from A to N but does not change the permissions.  It gives the following error:

PS C:\IT\SCRIPTS> .\test3.ps1
ERROR: No mapping between account names and security IDs was done.
PS C:\IT\SCRIPTS>

Open in new window

0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Did you apply the same correction as stated in http:#a40880250 ? The batch file had the same issue.
0
ITPro44Author Commented:
I did, however, I failed to save the changes.  My mistake.  It's working now.  

This script above only processes files with the Attribute set to "A".  Is the script below the proper way to process all .JPG files no matter if the attribute is "A" or "N"?

Get-ChildItem E:\Data -include *.jpg -recurse |
  % {
    xcacls "$($_.FullName)" /P "Authenticated Users:R" "Domain\Administrator:F" "SYSTEM:F" "Photo Admins:F" /Y
    $_.Attributes = $_.Attributes -bxor [System.IO.FileAttributes]::Archive
  }

Open in new window

0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Yes.
0
ITPro44Author Commented:
great, thanks.  I like that this powershell script identifies and processes each file in one swoop.

One other question I have - I currently don't have a way for this script to notify me if an error occurs.  Do have any suggestions?
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Since this code is using a mixture of external and internal commands, logging is a bit more demanding. I assume you've stored the code in a script file, and run it from PowerShell prompt. Then something like
  .\script.ps1 > log.txt 2>&1
will log both success and error messages, as you would see them on the screen (without colour, of course)
0
ITPro44Author Commented:
I'd like to have an email generated that will send the job status (success or error) with the log attached.  Is this possible?  Perhaps I should create another question for this.  Let me know if that is the best route.
0
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
That's a different question, and a more difficult one. Sending a mail is easy, but detecting success and failure needs some work.
0
ITPro44Author Commented:
Thanks for all the help.  I'll create a new question for this.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.