[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Count and Move only Paired files

Posted on 2011-10-15
61
Medium Priority
?
434 Views
Last Modified: 2012-05-12
I have an odd situation where i need to be able to both count and move as needed .rtf and .pdf files that are mirrored pairs of each other.  

in other words, the folder will contain give or take the following:

file1.rtf
file1.pdf
file2.rtf
file2.pdf
file3.rtf
file4.rtf
file4.pdf

and so on.  the problem is the file3.rtf, which for some reason didn't get mirrored correctly.  

I would ideally like to be able to run a script against the folder that will count the pairs and singles, then accept input to move them to a destination chosen from a displayed list.  

the ability to move either the pairs or the singles by input or flag would be pretty handy as well.

I know i'm asking alot.

Thanks in advance for your time.

Mark Holbert
0
Comment
Question by:mholbert
  • 24
  • 12
  • 12
  • +3
61 Comments
 
LVL 59

Expert Comment

by:Bill Prew
ID: 36973506
Could there ever be any other files in the folder with the same name, but a different extension?

Are all the files in a single folder, or are there subfolders involved?

~bp
0
 

Author Comment

by:mholbert
ID: 36973539
all the files in the folder will be either paired as .rtf/.pdf or .doc/.pdf.  There are two input folders but they are each only one level deep.  The input folders exist as below.

INPUT
  TRAN
  TRANDOC
 

There shouldn't be any files in there that are anomylous with the possible exception of an odd rename that might result in a .doc1 or .rtf1.  those would be ignored ideally.

my plan was to take whatever solution was delivered and morph it to address both input folders seperately, but i suppose it could address both folders at the same time.  

There are additional folders inside othe INPUT folder that would not need to be affected.

0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 36973552
But we are only looking for RTP/PDF pairs, right?  So if there was a file1.pdf and a file1.doc, for this script you'd want that flagged as a "single" since there is no RTF file?

~bp
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:mholbert
ID: 36973605
i guess actually im looking for rtf/pdf and doc/pdf pairs in seperate folders.   TRAN would only have rtf/pdf pairs and TRANDOC only doc/pdf pairs.

ideally it would count and allow movement of both types of pairs, and only avoid the single .rtf, .doc, or possible .doc1 or .rtf1 files.  the singles could just stay put as they will either need duplication or special handling in the case of the .doc1 and .rtf1 files.

MH
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36973984
Before I would code anything like this I have some questions:

1) Each .pdf file should have a samename.rtf or samename.doc file, NEVER both right?
2) If the .pdf file has no matches for the samename.rtf or samename.doc file then no action need be taken with the file?
3) The file pairs can be in different folders under the INPUT folder, so one file of the pair may be in one folder, but the other may be in another folder also under the INPUT dir?
4) You wish to count the pairs/singles? what will this let you do exactly? If it's important, then I assume that one pair counts as one instead of two?
5) For moving the files, do they need to keep the folder structure at all, or could all pairs be moved into the same folder?

That should be all for now, quite an interesting thing you have to do.
0
 

Author Comment

by:mholbert
ID: 36974013
1.  correct.
2.  correct. the more likely possibility would be that there would be a .doc or .rtf without a samename.pdf
3.  incorrect.  the TRAN input folder will contain matching pairs of .rtf and .pdf files, and the TRANDOC input folder will contain matching pairs of .doc and .pdf files.  There should be no cross folder pairing, and in fact consideration of that would cause problems, as there is a possibility of the samename.doc actually containting different information than the samename.rtf
4.  the count would be the first step, and would ideally allow a second step of moving either the pairs or the singles, depending on the operation required.  one pair would indeed count as one.  long story regarding the what i'm trying to do, and of course the original idea has gotten more complex as i am playing through it.  i am crunching transcribed medical documents through an engine that inserts them into a database and am running into orphaned single files at a high enough rate that it is making the testing process arduous.  When i push 10,000 files into the structure overnight, it is a pain when in the morning, 8000 got paired up appropriately, and for whatever reason 2000 did not.  i have to manually hunt down the 2000 and remove them before i can process the 8000 pairs (16000 files).
5.  all pairs could be moved into the same folder, ideally a folder defined as a variable for easy changing at the script level.

Thanks so much for your energy on this.  

MH
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36974041
Ah OK i'm with you I think ;)

Going to grab some food and maybe catch a film, if no one has given you a solution by the time I am back I may whip something up for you!
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36974076
interesting one.... Havent time to look at the mo. But listening to see where it goes.   my thoughts were dir listing sorted by name, record if there is a  pdf or doc or rtf before name changes. If name changed and only pdf or rtd/ doc then call routine a otherwise routine b...
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36974438
Okay try this function below. Read carefully the next couple of lines.

Usage is as follows:

MovePairs "C:\INPUT\TRAN" "rtf,pdf" "C:\INPUT\OUTPUT" 0

So it goes:

MovePairs <Input Dir> <File Extensions seperated by comma> <Output Dir> <0 for Move files, anything else for Read Only>

Function code needs to be loaded BEFORE you call the function, i.e. it would need to sit at the top of any further script code you run.

Put in some basic error handling, such as breaking if not all values are being called in the Function, and also not processing empty collections etc. Also if one of the files already exists in the target directory (file with the same name) then both files are not moved.

Also liberally put around some Write-Hosts for basic echoing back to host with information.

The idea behind this function is you can simply call it with the right folder and file extension arguments multiple times.

Code:

Function MovePairs
{
	param
	(
		$FolderPath,
		$FilePair,
		$MoveDir,
		$ReadOnly
	)
	
	if ($FolderPath -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Folder Path provided" ; break}
	if ($FilePair -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No FilePair provided" ; break}
	
	$File1 = $FilePair.Split(",")[0]
	$File2 = $FilePair.Split(",")[1]
	
	if ($File1 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File1 extension" ; break}
	if ($File2 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File2 extension" ; break}
	if ($MoveDir -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No MoveDirectory provided" ; break}
	if ($ReadOnly -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Readonly Mode not set" ; break}
	
	$FolderContent = Get-ChildItem $FolderPath\* -Include @("*.$File1","*.$File2")
	
	if ($FolderContent.Count -gt 0)
	{
	
		$FolderGrouping = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 2}
		
		if ($FolderGrouping.Count -gt 0)
		{
		
			foreach ($Grouping in $FolderGrouping)
			{
				Write-Host -ForeGroundColor "Yellow" $Grouping.Group
				if ($ReadOnly -eq 0)
				{
					Try
					{
						$Error = 0
						
						foreach ($GroupingFile in $Grouping.Group)
						{
							$MoveFilePath = $MoveDir + "\" + $GroupingFile.Name
							
							if (Test-Path $MoveFilePath)
							{
								$Error = 1
							}
						}
					}
					Catch [Exception]
					{
						Write-Host $($_.Exception)
					}
					Finally
					{
						if ($Error -eq 0)
						{
							$Grouping.Group | % {Move-Item $_ $MoveDir}
						}
						else
						{
							Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "One or more files exists in target path skipping moves"
						}
					}
				}
			}
		
		}
		else
		{
			Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Item pairs found"
			break
		}
	
	}
	else
	{
		Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No items found that match the extensions provided"
		break
	}
	
}

Open in new window

0
 

Author Comment

by:mholbert
ID: 36974481
nice, i will try it shortly and let you know.  thx.  MH
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36974558
Here was a quick batch file version I knocked up ... haven't had chance to test it so may or not be typos in there.  Will look more if you want to go batch route instead of powershell.

Steve
@echo off
set doublemovedir="c:\move"
set singlemovedir="C:\move"

REM First call sub for rtf, then doc (or any others put below)
REM Loop through all doc or rtf files in that dir and look for matching PDF 
REM and count it or move it as per third parameter

call :searchfiles rtf "c:\input\tran" count
call :searchfiles doc "c:\input\trandoc" count

set /p moveit=Move files Y/N?

if /i "%moveit%"=="Y" (
  call :searchfiles rtf "c:\input\tran" move
  call :searchfiles doc "c:\input\trandoc" move
)

exit /b

:searchfiles
  set single=0
  set double=0
  pushd "%~2"
    echo Looking for %1  files in "%~2" to %3
    for %%F in (*.%1) do (
      if exist "%%nF.pdf" (
        echo Found %%~nxF and %%~nF.pdf
        set /a double=double + 1
        if %3==move (
          move "%%~nxF" %doublemovedir%
          move "%%~nF.pdf" %doublemovedir%
        )
      ) ELSE (
        echo Found %%~nxF but NO PDF file
       set /a single=single + 1
       if %3==move move "%%~nxF" %singlemovedir%
      )
    )
  popd
  echo There were %double% doubles and %single% singles of %1 in the %~2 dir.
exit /b

Open in new window

0
 

Author Comment

by:mholbert
ID: 36974635
sadly, powershell is on my list of wanna learns, but it hasn't happened yet.  no time like the present i suppose.  if you can give me a quick 1,2,3 i will give it a whirl.  
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36974694
If you want to run the PowerShell stuff then just copy and paste my code into a PowerShell window (in PowerShell right click will paste).

Hit return a couple of times until you are back at the prompt and then that code will be considered as "run". Now you can call the function with your arguments, as per my example:

MovePairs "C:\INPUT\TRAN" "rtf,pdf" "C:\INPUT\OUTPUT" 1

Change the last argument to 0 when you want to move the files and not just report them back to the host. Also change the folder paths etc to fit your needs.

If you need to schedule this you can put into a .ps1 script file and call it with a .bat file on a scheduled task, however i'd get familiar with running it manually first.
0
 

Author Comment

by:mholbert
ID: 36974728
i pasted your code into PowerShell and it threw the following error 83 times, incrementing the value by 1 each time.  then it appears it listed the function.  

PS C:\Users\mholbert> 1:
Unexpected token ':' in expression or statement.
At line:1 char:3
+ 1: <<<<
    + CategoryInfo          : ParserError: (::String) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : UnexpectedToken
0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 36974848
Okay, couldn't get back to this today, but here's my BAT approach. Hopefully the comments explain what's going on, it's basically driven by the call to :DoDir, where you pass the folders and extensions needed.  It will scan the folder looking for the matches on the extensions passed, display the counts, and then ask for what to move.

I've used a simple way to indicate which files to move, but you have to get the idea.  It will prompt you with:

What would you like to move? (YN=pairs, NY=singles, YY=both. NN=none)

Basically your response needs to be a 2 character response, ot Y
s and N's.  The first character is the pairs move flag, and the second character is the singles move flag.  If the flag is a Y, then those files are moved, If it's a N, then no move is done for that set of files.

@echo off
setlocal EnableDelayedExpansion

REM Process folders desired, passing the source extension, and result extension, and the move location for pairs/singles
call :DoDir "c:\input\tran" "rtf" "pdf" "c:\dest\pairs" "c:\dest\singles"
call :DoDir "c:\input\trandoc" "doc" "pdf" "c:\dest\pairs" "c:\dest\singles"
goto :EOF

REM Subroutine to look for pairs/singles, and then move as desired
:DoDir [scan-folder] [source-ext] [dest-ext] [pairs-dest-folder] [singles-dest-folder]
  REM Display folder and extensions being worked
  echo .
  echo Processing dir:[%~1, exts:[%~2, %~3]

  REM Make sure the destination folders for pairs/singles exist
  if not exist "%~4\" md "%~4\"
  if not exist "%~5\" md "%~5\"

  REM Switch to the folder to scan
  pushd "%~1"

  REM Get a list of the file names (no extension) for each extension passed in
  (for %%A in (*.%~2) do echo %%~nA)>_%~2.txt
  (for %%A in (*.%~3) do echo %%~nA)>_%~3.txt

  REM Get a count for pairs, and singles by compairing the lists of files
  for /F %%A in ('findstr ^<_%~3.txt /I /G:_%~2.txt ^| find /V /C ""') do set Pairs=%%A
  for /F %%A in ('findstr ^<_%~2.txt /V /I /G:_%~3.txt ^| find /V /C ""') do set Singles=%%A

  REM Display stats, and ask for any moves desired
  echo Pairs:[%Pairs%], Singles:[%Singles%]
  set /P "Move=What would you like to move? (YN=pairs, NY=singles, YY=both. NN=none): "

  REM Do we want to move the pairs?
  if /I "%Move:~0,1%" EQU "Y" (
    for /F %%A in ('findstr ^<_%~3.txt /I /G:_%~2.txt') do (
      move "%%~A.%~2" "%~4" >NUL
      move "%%~A.%~3" "%~4" >NUL
    )
  )

  REM Do we want to move the singless?
  if /I "%Move:~1,1%" EQU "Y" (
    for /F %%A in ('findstr ^<_%~2.txt /V /I /G:_%~3.txt') do (
      move "%%~A.%~2" "%~5" >NUL
    )
  )

  REM Remove temp file lists
  del _%~2.txt
  del _%~3.txt

  REM Back to original folder
  popd
  goto :EOF

Open in new window

0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36975620
mholbert, when you pasted my code in did you paste in the line numbers too? :P

Line numbers are just for illustration here, please try copying/pasting the code in without the line numbers at the start!
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36975681
This was tested version of what I did.... one minor typo and few slight changes since.  Like Bill's extra choices and slicker counting mind!


@echo off
setlocal enabledelayedexpansion
set doublemovedir="C:\4. WIP\EE\27398422\movedouble"
set singlemovedir="C:\4. WIP\EE\27398422\movesingle"
set debug=0

mkdir %doublemovedir% 2>NUL
mkdir %singlemovedir% 2>NUL

REM First call sub for rtf, then doc (or any others put below)
REM Loop through all doc or rtf files in that dir and look for matching PDF 
REM and count it or move it as per third parameter

call :searchfiles rtf "C:\4. WIP\EE\27398422\tran" count
call :searchfiles doc "C:\4. WIP\EE\27398422\trandoc" count

set /p moveit=Move files [Y/N]?

if /i "%moveit%"=="Y" (
  call :searchfiles rtf "C:\4. WIP\EE\27398422\tran" move
  call :searchfiles doc "C:\4. WIP\EE\27398422\trandoc" move
)

exit /b

:searchfiles
  set single=0
  set double=0
  pushd "%~2"
    echo Looking for %1 files in "%~2" to %3
    for %%F in (*.%1) do (
      if exist "%%~nF.pdf" (
        if %debug%==1 echo Found %%~nxF and %%~nF.pdf
        set /a double=double + 1
        if %3==move (
          move "%%~nxF" %doublemovedir% >NUL
          move "%%~nF.pdf" %doublemovedir%> NUL
        )
      ) ELSE (
        if %debug%==1 echo Found %%~nxF but NO %%~nF.pdf file
       set /a single=single + 1
       if %3==move move "%%~nxF" %singlemovedir% >NUL
      )
    )
  popd
  echo There were %double% doubles and %single% singles of %1 in the %~2 dir.
exit /b

Open in new window

0
 

Author Comment

by:mholbert
ID: 36975911
Wow, you guys are impressive.  After a half a nights sleep i am back at it.  I'm testing Bill's example simply because it is the format i am most familiar with.  I will eventually test all of them.  as written, this one moved most everything appropriately.  it did throw 'system cannot find file' errors, but i am looking into the why's of that.  

I tried rebuilding the top a little to allow for easier manipulation of the folder paths and clearly don't understand the syntax well enough.  I changed the top of it to the following:  

 
REM  BillPrew Version

@echo off
setlocal EnableDelayedExpansion

Set RTFDoDir = "\\ferret\ferret\ERRORS\TRAN\20111014"
Set RTFPairsDir = "\\ferret\ferret\ERRORS\TRAN\pairs"
Set RTFSinglesDir = "\\ferret\ferret\ERRORS\TRAN\singles"

Set DOCDoDir = "\\ferret\ferret\ERRORS\TRANDOC\20111014"
Set DOCPairsDir = "\\ferret\ferret\ERRORS\TRANDOC\pairs"
Set DOCSinglesDir = "\\ferret\ferret\ERRORS\TRANDOC\singles"

REM Process folders desired, passing the source extension, and result extension, and the move location for pairs/singles
call :DoDir %RTFDoDir% "rtf" "pdf" %RTFPairsDir% %RTFSinglesDir%
call :DoDir %DOCDoDir% "doc" "pdf" %DOCPairsDir% %DocSinglesDir%
goto :EOF

Open in new window


and got the following error in the command window:

 
F:\MHolbert\Desktop\TranCatchup>REM  BillPrew Version
.
Processing dir:[rtf, exts:[pdf, ]
The system cannot find the path specified.
FINDSTR: No search strings
FINDSTR: No search strings
Pairs:[0], Singles:[0]
What would you like to move? (YN=pairs, NY=singles, YY=both. NN=none):

Open in new window


It should be noted that it clearly works as designed and i am just pushing the envelope a little.  
0
 

Author Comment

by:mholbert
ID: 36976907
I went back to the original version and it works, for the most part.  it displays 'The System cannot find the file specified' for quite a few files that it seems unwilling to pickup.  i can manuallystrip them out and they move without incident.  thanks again.  MH
0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 36977038
==> I tried rebuilding the top a little to allow for easier manipulation of the folder paths and clearly
==> don't understand the syntax well enough.  I changed the top of it to the following:  

Based on the changes you showed there I wouldn't expect the error you got.  Looking at the first ECHO from DoDir:

Processing dir:[rtf, exts:[pdf, ]

Open in new window

It seems to indicate the first parm wasn't passed.  Did you change anything else in that script?

Also, might be worth trying to put all the parms in quotes, as in:

call :DoDir "%RTFDoDir%" "rtf" "pdf" "%RTFPairsDir%" "%RTFSinglesDir%"
call :DoDir "%DOCDoDir%" "doc" "pdf" "%DOCPairsDir%" "%DocSinglesDir%"

Open in new window

~bp
0
 

Author Comment

by:mholbert
ID: 36977578
I have a follow up question.  i just realized i need to move finished pairs (both types) from one folder to another every few minutes.

I have some logic to perform a move operation every few seconds, how can i add your logic into it to clean it up and make it simple.

the move every X seconds is here.  
set BaseDir=F:\TranCatchupFiles\TranDOCLoad
set DestDir=\\Ferret\Ferret\input\TRANDoc
set Chunk=10
set Delay=120000
Set REPS=0
Set Cycle=%Delay%

REM Main loop to look for the next group of files to move
:Loop
  Set /a REPS += 1
  set Count=0
  REM Process up to group size of files
  for %%A in ("%BaseDir%\*.doc") do (
    set /A Count+=1
    if !Count! GTR %Chunk% goto :Wait
rem echo Moving [%%~A] to [%DestDir%]
    move "%%~A" "%DestDir%"
  )

REM Delay for the desired time and then look again
:Wait
echo ==========================
echo Moving [%BaseDir%] 
echo   to   [%DestDir%]
echo DOC [%REPS% x %Chunk% @ %Cycle%]
echo ==========================
ping 1.1.1.1 -n 1 -w %Delay%
goto :Loop

Open in new window


i would like to incorporate the ability to only move matched pairs of .doc/.pdf and .rtf/.pdf files.

i run the script above manually while i am catching up the backlog of files.  once that is done, i would like to run the eventual script from a scheduled task.

0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36977817
are we just talking bills script here..... Doesnt bother me but not going to amend if not wanted!

Steve
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36977822
i assume what you are saying then.... And i might do this anyway with my script is:

a) automatic run with move any pairs it finds to pairs dir.
b) manual run to move pairs or move singles - do you want it to run a program for each single, move it to a dir for it to pickup automatically, or leave them behind for you to process later?

0
 
LVL 11

Expert Comment

by:paultomasi
ID: 36978629
mholbert

Just stumbled upon this question.

Please do not close this question just yet.

It's an interesting question and I do not think it has been addressed properly yet....

0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36978804
I wondered where you had got to Paul!
0
 
LVL 11

Expert Comment

by:paultomasi
ID: 36978913
Been busy with family....

Trouble is, I could easily spend 20 hrs a day at the computer and still not get everything done!

(Perhaps I should have taken up horsegrooming or morris dancing instead!)
0
 

Author Comment

by:mholbert
ID: 36978957
I was referencing Bill's script, but i am happy to proceed in whatever way appears appropriate to the situation.  

at present i would want to only pickup pairs and leave singles behind.

thx.  MH
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36979040
You should at least test my one if you can mholbert, I know PowerShell might not be your home script as it were but you may be surprised :)
0
 

Author Comment

by:mholbert
ID: 36979052
I plan on it as soon as i am no longer in crisis mode on a couple issues.  I will play through all the scripts as this week progresses and PowerShell is one of the things i have been meaning to dig deeper into for quite some time so i look forwards to finding the time.
0
 
LVL 11

Expert Comment

by:paultomasi
ID: 36979062
mholbert

Just out of curiosity, is this program (which you have asked us to write) going to be run on a routine basis, or just this once (to solve your current problem only)?
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36979245
OK here is my updated script.  I haven't re-done it using quicker count methods etc. as I tried it on 10000 files and was only taking a few secs anyway so seemed OK to me, though I know there are quicker methods (Bill / no doubt Paul shortly!)

It can be called with

batchfile.cmd AUTO

to auto-process any paired files into the relevant directory.  A log file will be kept of how many are copied and any error messages.

If you change the line set debug=0 to set debug=1 it will also display each name as it copies them, or will add those to the log too.

Run it without AUTO and it will count the singles and doubles and present you with two choices "move pairs" and "move singles".  If you say "Y" or "y" to either it will do what it says...

If you want to incorporate this into your existing script then where you have the MOVE command you could just make it along the lines of :

    call moveit.cmd AUTO

but you could also bring it all into one, though then more messy to run the whole thing on a schedule etc.

Steve
@echo off

setlocal enabledelayedexpansion

set inputpath=C:\4. WIP\EE\27398422
set doublemovedir="C:\4. WIP\EE\27398422\movedouble"
set singlemovedir="C:\4. WIP\EE\27398422\movesingle"
set logfile="C:\4. WIP\EE\27398422\log.txt"

mkdir %doublemovedir% 2>NUL
mkdir %singlemovedir% 2>NUL

REM Set debug to 1 to show progress on screen of each move, or will go to log file for AUTO
set debug=0

REM Call batch file with parameter of AUTO to have it automove the paired files only
REM No need to go through count routines if automoving but will record to log file

if /i %~1=AUTO (
    (call :searchfiles rtf "%inputpath%\tran" pairs
     call :searchfiles doc "%inputpath%\trandoc" pairs) >> %logfile%
) ELSE (

  REM First call sub for rtf, then doc (or any others put below)

  call :searchfiles rtf "%inputpath%\tran" count
  call :searchfiles doc "%inputpath%\trandoc" count

  set /p moveP=Move pairs [Y/N]?
  set /p moveS=Move singles [Y/N]?

  if /i "%moveP%"=="Y" (
    call :searchfiles rtf "%inputpath%\tran" pairs
    call :searchfiles doc "%inputpath%\trandoc" pairs
  )

  if /i "%moveS%"=="Y" (
    call :searchfiles rtf "%inputpath%\tran" singles
    call :searchfiles doc "%inputpath%\trandoc" singles
  )

)
exit /b


  REM Loop through all doc or rtf files in that dir and look for matching PDF 
  REM and count it or move it as per third parameter

:searchfiles
  set single=0
  set double=0
  pushd "%~2"
    echo Looking for %1 files in "%~2" to %3
    for %%F in (*.%1) do (
      if exist "%%~nF.pdf" (
        if %debug%==1 echo Found %%~nxF and %%~nF.pdf
        set /a double=double + 1
        if %3==pairs (
          move "%%~nxF" %doublemovedir% >NUL
          move "%%~nF.pdf" %doublemovedir%> NUL
        )
      ) ELSE (
        if %debug%==1 echo Found %%~nxF but NO %%~nF.pdf file
       set /a single=single + 1
       if %3==singles move "%%~nxF" %singlemovedir% >NUL
      )
    )
  popd
  echo There were %double% doubles and %single% singles of %1 in the %~2 dir.
exit /b

Open in new window

0
 

Author Comment

by:mholbert
ID: 36983178
Paul.  The original idea was to be used to cleanup an issue.  Once i saw the functionality i realized it would be helpful in the ongoing process of converting and processing files into the database. i think that makes the answer both.

Chris.  Good Point.  I started with yours originally and then aborted when i got the first error.  Now that i reread the info you provided, it would appear it does both the short term and long term goals all in one move.  I will be playing with it in a matter of minutes.

Thanks to all of you that have contributed ideas and code.  The hardest part has been choosing which code to go with, as i'm sure they all have interesting advantages and disadvantages.  I will post updates as it starts to come together.  

Mark
0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 36983274
Rather than run the BAT continuously with a delay in the loop, I'd suggest using the Windows Scheduler to run the script at the desired frequency.

At this point it's not real clear to me what script you are building upon, so before I make changes to mine I'll wait for you to direct me on my specific approach if needed.  There are many ways to solve the problem...

~bp
0
 

Author Comment

by:mholbert
ID: 36983277
I just tested Chris's script and it ran perfectly.  I must say i also like Steve's idea of logging everything so i am going to test that now.  

Mark
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36984321
Good stuff, if you need any changes made on my script let me know.
0
 

Author Comment

by:mholbert
ID: 36990066
Chris, i would be great if it would log the transfers and display a count of the files after the listing.

It would additionally be helpful if it would recycle the singles that inevitably remain.  

in the end it would move the pairs, send the singles back to the <inputdir> and log to <logpath> input variable i guess.

when called from within a bat, if i needed to move pairs from inputdirs a, b, c to destdirs a, b, c, would i just  put the syntax below in the bat file?  does it need any delay between statements or does it run them all at once?  


MovePairs "\\op2\FERRET\INPUT\TRANDOC" "doc,pdf" "\\ferret\ferret\INPUT\TRANDOC" 0
MovePairs "\\op2\FERRET\INPUT\TRAN" "rtf,pdf" "\\ferret\ferret\INPUT\TRAN" 0
MovePairs "\\op2\ferret\INPUT\LAB" "rtf,pdf" \\ferret\ferret\INPUT\LAB 0

Open in new window

0
 

Author Comment

by:mholbert
ID: 36990166
I forgot to include that the singles wouldn't need to be moved unless they had been in place for X minutes (probably 30) and they would need to take a short trip to another folder before returning to the starting point so they would be collected as 'fresh'.
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36995059
Alright, i've changed the function to include counts, and also the ability to move single and double files to different folders.

The usage has changed a little, you need to put in:

MovePairs <InputDir> <FilePair> <SingleMoveDirectory> <DoubleMoveDirectory> <ReadOnly>

This is an example:

MovePairs "C:\INPUT\TRAN" "rtf,pdf" "C:\INPUT\SINGLE" "C:\INPUT\OUTPUT" 0

This is the function code:

Function MovePairs
{
	param
	(
		$FolderPath,
		$FilePair,
		$MoveDirSingle,
		$MoveDirDouble,
		$ReadOnly
	)
	
	#Count Array
	$CountArray = @()
	$CountArray = "" | Select TotalCount, SingleCount, SingleCountMoved, SingleCountFailed, DoubleCount, DoubleCountMoved, DoubleCountFailed
	
	if ($FolderPath -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Folder Path provided" ; break}
	if ($FilePair -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No FilePair provided" ; break}
	
	$File1 = $FilePair.Split(",")[0]
	$File2 = $FilePair.Split(",")[1]
	
	if ($File1 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File1 extension" ; break}
	if ($File2 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File2 extension" ; break}
	if ($MoveDirSingle -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Single Directory provided" ; break}
	if ($MoveDirDouble -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Double Directory provided" ; break}
	if ($ReadOnly -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Readonly Mode not set" ; break}
	
	$FolderContent = Get-ChildItem $FolderPath\* -Include @("*.$File1","*.$File2")
	
	if ($FolderContent.Length -gt 0)
	{
	
		$FolderGroupingSingle = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 1}
		$FolderGroupingDouble = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 2}
		
		$CountArray.TotalCount = ($FolderGroupingSingle.Length + $FolderGroupingDouble.Length)
		$CountArray.SingleCount = $FolderGroupingSingle.Length
		$CountArray.DoubleCount = $FolderGroupingDouble.Length
		
		#Loop through Doubles
		if ($FolderGroupingDouble.Length -gt 0)
		{
		
		$DoubleCountMoved = 0
		$DoubleCountFailed = 0
		
			foreach ($GroupingDouble in $FolderGroupingDouble)
			{
				Write-Host -ForeGroundColor "Yellow" $GroupingDouble.Group
				if ($ReadOnly -eq 0)
				{
					Try
					{
						$ErrorDouble = 0
						
						foreach ($GroupingDoubleFile in $GroupingDouble.Group)
						{
							$MoveFilePathDouble = $MoveDirDouble + "\" + $GroupingDoubleFile.Name
							
							if (Test-Path $MoveFilePathDouble)
							{
								$ErrorDouble = 1
							}
						}
					}
					Catch [Exception]
					{
						Write-Host $($_.Exception)
					}
					Finally
					{
						if ($ErrorDouble -eq 0)
						{
							$GroupingDouble.Group | % {Move-Item $_ $MoveDirDouble}
							$DoubleCountMoved++
						}
						else
						{
							Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "One or more files exists in target path skipping moves"
							$DoubleCountFailed++
						}
					}
				}
			}
		
		}
		
		$CountArray.DoubleCountMoved = $DoubleCountMoved
		$CountArray.DoubleCountFailed = $DoubleCountFailed
		
		#Loop through Singles
		if ($FolderGroupingSingle.Length -gt 0)
		{
		
		$SingleCountMoved = 0
		$SingleCountFailed = 0
		
			foreach ($GroupingSingle in $FolderGroupingSingle)
			{
				Write-Host -ForeGroundColor "Yellow" $GroupingSingle.Group
				if ($ReadOnly -eq 0)
				{
			
					$ErrorSingle = 0
					
					$GroupingSingleFile = $GroupingSingle.Group[0]
					
					$MoveFilePathSingle = $MoveDirSingle + "\" + $GroupingSingleFile.Name
					
					if (Test-Path $MoveFilePathSingle)
					{
						$ErrorSingle = 1
						Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "One or more files exists in target path skipping move"
						$SingleCountFailed++
					}
					else
					{
						$GroupingSingle.Group | % {Move-Item $_ $MoveDirSingle}
						$SingleCountMoved++
					}
					
				}
			}
		}
		
		$CountArray.SingleCountMoved = $SingleCountMoved
		$CountArray.SingleCountFailed = $SingleCountFailed
	
	}
	else
	{
		Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No items found that match the extensions provided"
		break
	}
	
	$CountArray | fl | Out-Host
	
}

Open in new window


Regarding your points on moving single files to a directory for a bit, how do you want this to work in the context of my script?

For the moving only single files that have been in place for more than 30 minutes, is this based on the Modified date or the Created date value? let me know I can easily add a check in or skip those files over all together.
0
 
LVL 11

Expert Comment

by:paultomasi
ID: 36995447
Please be aware I am ready to comment on this question!
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36995629
Ah man, re-read what you said and made some more changes! Please read all the below carefully.

Logging Function
Added a Logging function that will export to LOG File (appends only) and also anything that goes to logfile also goes to host if you are running from the powershell window.

Counts
Updated the counts at the end so they work better now.

Usage
To work with the logging you need to pass it a log file location, so the new usage is as follows:

MovePairs <InputDir> <FilePair> <SingleMoveDirectory> <DoubleMoveDirectory> <LogFile> <ReadOnly>

Example:

MovePairs "C:\INPUT\TRAN" "rtf,pdf" "C:\INPUT\SINGLE" "C:\INPUT\OUTPUT" "C:\INPUT\log.txt" 0

Input on how to handle single files
Please advise on if you want changes to the way single files are handled, also need to know if this 30 min thing you want is based on created date or modified date.

Running in a script
You should put everything into a .ps1 script with the function at the top of the code, then just call the .ps1 script from the batch file. PowerShell is smart enough to call things in order, but needs to use PowerShell.exe to execute the code, so it won't skip over lines if you don't include any delay.

This is the most updated version now:

Function MovePairs
{
	param
	(
		$FolderPath,
		$FilePair,
		$MoveDirSingle,
		$MoveDirDouble,
		$LogFile,
		$ReadOnly
	)
	
	#Count Array
	$CountArray = @()
	$CountArray = "" | Select TotalCount, SingleCount, SingleCountMoved, SingleCountFailed, DoubleCount, DoubleCountMoved, DoubleCountFailed
	
	if ($FolderPath -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Folder Path provided" ; break}
	if ($FilePair -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No FilePair provided" ; break}
	
	$File1 = $FilePair.Split(",")[0]
	$File2 = $FilePair.Split(",")[1]
	
	if ($File1 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File1 extension" ; break}
	if ($File2 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File2 extension" ; break}
	if ($MoveDirSingle -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Single Directory provided" ; break}
	if ($MoveDirDouble -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Double Directory provided" ; break}
	if ($LogFile -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Logfile not set" ; break}
	if ($ReadOnly -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Readonly Mode not set" ; break}
	
	#Logging Function
	Function Logging
	{
		param
		(
			$Colour
		)
		
		$Date = (Get-Date).ToString("dd/MM/yyyy HH:mm:ss")
		$String = $Date + " : " + $input
		
		$String | Out-File $LogFile -Append
		if ($Colour)
		{
			Write-Host -ForeGroundColor $Colour $String
		}
		else
		{
			Write-Host $String
		}
	}
	
	#Export info about current run to log file
	"Folder Path Entered: $FolderPath" | Logging
	"File Pair: $FilePair" | Logging
	"Move Directory Single: $MoveDirSingle" | Logging
	"Move Directory Double: $MoveDirDouble" | Logging
	"Log File: $LogFile" | Logging
	"Read Only: $ReadOnly" | Logging
	
	[array]$FolderContent = Get-ChildItem $FolderPath\* -Include @("*.$File1","*.$File2")
	
	if ($FolderContent.Length -gt 0)
	{
	
		[array]$FolderGroupingSingle = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 1}
		[array]$FolderGroupingDouble = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 2}
		
		$CountArray.TotalCount = ($FolderGroupingSingle.Length + $FolderGroupingDouble.Length)
		$CountArray.SingleCount = $FolderGroupingSingle.Length
		$CountArray.DoubleCount = $FolderGroupingDouble.Length
		
		#Loop through Doubles
		if ($FolderGroupingDouble.Length -gt 0)
		{
		
		$DoubleCountMoved = 0
		$DoubleCountFailed = 0
		
			foreach ($GroupingDouble in $FolderGroupingDouble)
			{
				$GroupingDouble.Group | Logging "Yellow"
				if ($ReadOnly -eq 0)
				{
					Try
					{
						$ErrorDouble = 0
						
						foreach ($GroupingDoubleFile in $GroupingDouble.Group)
						{
							$MoveFilePathDouble = $MoveDirDouble + "\" + $GroupingDoubleFile.Name
							
							if (Test-Path $MoveFilePathDouble)
							{
								$ErrorDouble = 1
							}
						}
					}
					Catch [Exception]
					{
						$($_.Exception) | Logging "Red"
					}
					Finally
					{
						if ($ErrorDouble -eq 0)
						{
							$GroupingDouble.Group | % {Move-Item $_ $MoveDirDouble ; "Moved Item $_" | Logging}
							$DoubleCountMoved++
						}
						else
						{
							"Move Failed as one or more files exist in destination path" | Logging "Red"
							$DoubleCountFailed++
						}
					}
				}
			}
		
		}
		
		$CountArray.DoubleCountMoved = $DoubleCountMoved
		$CountArray.DoubleCountFailed = $DoubleCountFailed
		
		#Loop through Singles
		if ($FolderGroupingSingle.Length -gt 0)
		{
		
		$SingleCountMoved = 0
		$SingleCountFailed = 0
		
			foreach ($GroupingSingle in $FolderGroupingSingle)
			{
				$GroupingSingle.Group | Logging "Yellow"
				
				if ($ReadOnly -eq 0)
				{
			
					$ErrorSingle = 0
					
					$GroupingSingleFile = $GroupingSingle.Group[0]
					
					$MoveFilePathSingle = $MoveDirSingle + "\" + $GroupingSingleFile.Name
					
					if (Test-Path $MoveFilePathSingle)
					{
						$ErrorSingle = 1
						"Move Failed as one or more files exist in destination path" | Logging "Red"
						$SingleCountFailed++
					}
					else
					{
						$GroupingSingle.Group | % {Move-Item $_ $MoveDirSingle ; "Moved Item $_" | Logging}
						$SingleCountMoved++
					}
					
				}
			}
		}
		
		$CountArray.SingleCountMoved = $SingleCountMoved
		$CountArray.SingleCountFailed = $SingleCountFailed
	
	}
	else
	{
		"No items found that match the extensions provided" | Logging "Red"
		break
	}
	
	"`n"
	"{0,-18} {1,0} {2,-2}" -f "TotalCount", ":", $CountArray.TotalCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCount", ":", $CountArray.SingleCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCountMoved", ":", $CountArray.SingleCountMoved | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCountFailed", ":", $CountArray.SingleCountFailed | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCount", ":", $CountArray.DoubleCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCountMoved", ":", $CountArray.DoubleCountMoved | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCountFailed", ":", $CountArray.DoubleCountFailed | Logging "Magenta"
	"`n"
	
}

Open in new window

0
 

Author Comment

by:mholbert
ID: 36995912
Chris

30 minutes is a guess on my part.  the problem i am solving is that the files are getting dropped into a folder by a robocopy script, and they should be duplicated as .pdf files within a few minutes ideally.  for some reason something occurs that causes it to miss files.  generally, you can just cut and paste the files into another folder for a few seconds, then put them back into the input folder and they will process correctly.  

this script is at the point where there should be pairs of files that need to move on their insertion into the database.  the left over singles just need to go back throgh for processing, and theoretically, if they don't get processed after a certain number of tries they would need to move to an error structure.  that sounds complicated.  

the creation time of the files could be months prior to their insertion, so the create and modify timestamps are useless.  is there a 'time written into this folder' value somewhere?  

i made a hold directory at \\op2\ferret\HOLD to allow for the temporary placement of files, or i suppose simply the delivery of singles that won't process for some reason.
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 36996028
Right you are, can you run this against one of your broken single files for me and let me know which one looks the best one to use for you? obv. change the file path to one in your system :)

$File = gci "C:\INPUT\TRAN\file1.pdf"
$File | Select CreationTime, LastWriteTime, LastAccessTime | fl

There are 3 timestamps on the files, one of them should hopefully be useful to us, created date might not be if the file is moved, but the others might help tell us when the file was moved. Alternatively you can't tell when a file is moved, I can't see any other datetime properties on the file that will assist us in all honesty.

Can we dump the single files into a folder, and then at the end of the script run shift them all again into the input folder again?
0
 

Author Comment

by:mholbert
ID: 36996086
that script returns the following:  

CreationTime   : 10/18/2011 3:03:39 PM
LastWriteTime  : 10/18/2011 3:03:39 PM
LastAccessTime : 10/19/2011 1:47:25 PM

From those i would have to pick LastAccessTime as the best one to work from.  

i think as long as the singels stay gone from the input folder long enough for it to clear its memory then they can go back in.  maybe the beginning of the script picks up the singles (from the previous run) and put them into the HOLD directory, and then the Pairs are moved, and the next singles go to HOLD. if it is run via scheduled task every X minutes it should serve its purpose.

thanks again for the energy expended on this.  

Mark
0
 

Author Comment

by:mholbert
ID: 36996126
Paul, I am not sure if i should be nervous or excited that you are about to comment.  Am i in trouble again?  

Mark

0
 
LVL 9

Accepted Solution

by:
chrismerritt earned 2000 total points
ID: 36996272
Alright, this should only move single files that are older than 30 minutes old on the last access time, it will ignore the files otherwise.

Also, it will move all the single files from the SingleDirectory to the InputDirectory at the start of the run, then any picked up on that run are moved into the SingleDirectory for the next run to pick up.

This will of course mean that the files will go around in an eternal loop though as it will move them to input, if they are destined to fail they will show up in the processing folder I guess, it will pick them up as single files, move back to the single directory, when then next run kicks off they will wind up back in the input directory again I guess.

Starting to have to feed it quite a lot of arguments now, would it be better to hardcode some of the folder paths in instead? i.e. the ones that never change. I assume you want to be able to say which folder to process against on the fly and the file extensions, but I guess the Input Directory and the Output Directory (for both Single/Double files) will remain static more or less?

Usage
Put your variables in this order:

            $FolderPath,
            $FilePair,
            $MoveDirSingle,
            $MoveDirDouble,
            $LogFile,
            $InputDir,
            $ReadOnly

Like this:

MovePairs "C:\INPUT\TRAN" "rtf,pdf" "C:\INPUT\SINGLE" "C:\INPUT\OUTPUT" "C:\INPUT\log.txt" "C:\INPUT\INPUT" 0

Function MovePairs
{
	param
	(
		$FolderPath,
		$FilePair,
		$MoveDirSingle,
		$MoveDirDouble,
		$LogFile,
		$InputDir,
		$ReadOnly
	)
	
	#Count Array
	$CountArray = @()
	$CountArray = "" | Select TotalCount, SingleCount, SingleCountMoved, SingleCountFailed, DoubleCount, DoubleCountMoved, DoubleCountFailed
	
	if ($FolderPath -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Folder Path provided" ; break}
	if ($FilePair -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No FilePair provided" ; break}
	
	$File1 = $FilePair.Split(",")[0]
	$File2 = $FilePair.Split(",")[1]
	
	if ($File1 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File1 extension" ; break}
	if ($File2 -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Missing File2 extension" ; break}
	if ($MoveDirSingle -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Single Directory provided" ; break}
	if ($MoveDirDouble -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Move Double Directory provided" ; break}
	if ($LogFile -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Logfile not set" ; break}
	if ($InputDir -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "No Input Directory provided" ; break}
	if ($ReadOnly -eq $Null) {Write-Host -ForeGroundColor "Red" -BackGroundColor "Black" "Readonly Mode not set" ; break}
	
	#Logging Function
	Function Logging
	{
		param
		(
			$Colour
		)
		
		$Date = (Get-Date).ToString("dd/MM/yyyy HH:mm:ss")
		$String = $Date + " : " + $input
		
		$String | Out-File $LogFile -Append
		if ($Colour)
		{
			Write-Host -ForeGroundColor $Colour $String
		}
		else
		{
			Write-Host $String
		}
	}
	
	#Export info about current run to log file
	"Folder Path Entered: $FolderPath" | Logging
	"File Pair: $FilePair" | Logging
	"Move Directory Single: $MoveDirSingle" | Logging
	"Move Directory Double: $MoveDirDouble" | Logging
	"Log File: $LogFile" | Logging
	"Input Directory: $InputDir" | Logging
	"Read Only: $ReadOnly" | Logging
	
	#Move files from HoldDirectory
	if ($ReadOnly -eq 0)
	{
		gci $MoveDirSingle | % {Move-Item $_.FullName $InputDir ; "Moved File from Holding Dir $_.FullName" | Logging "Yellow"}
	}
	
	[array]$FolderContent = Get-ChildItem $FolderPath\* -Include @("*.$File1","*.$File2")
	
	if ($FolderContent.Length -gt 0)
	{
	
		[array]$FolderGroupingSingle = $FolderContent | ? {$_.LastAccessTime -lt (Get-Date).AddMinutes(-30)} | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 1}
		[array]$FolderGroupingDouble = $FolderContent | Group-Object -Property {$_.Name.SubString(0, ($_.Name.LastIndexOf(".")))} | ? {$_.Count -eq 2}
		
		$CountArray.TotalCount = ($FolderGroupingSingle.Length + $FolderGroupingDouble.Length)
		$CountArray.SingleCount = $FolderGroupingSingle.Length
		$CountArray.DoubleCount = $FolderGroupingDouble.Length
		
		#Loop through Doubles
		if ($FolderGroupingDouble.Length -gt 0)
		{
		
		$DoubleCountMoved = 0
		$DoubleCountFailed = 0
		
			foreach ($GroupingDouble in $FolderGroupingDouble)
			{
				$GroupingDouble.Group | Logging "Yellow"
				if ($ReadOnly -eq 0)
				{
					Try
					{
						$ErrorDouble = 0
						
						foreach ($GroupingDoubleFile in $GroupingDouble.Group)
						{
							$MoveFilePathDouble = $MoveDirDouble + "\" + $GroupingDoubleFile.Name
							
							if (Test-Path $MoveFilePathDouble)
							{
								$ErrorDouble = 1
							}
						}
					}
					Catch [Exception]
					{
						$($_.Exception) | Logging "Red"
					}
					Finally
					{
						if ($ErrorDouble -eq 0)
						{
							$GroupingDouble.Group | % {Move-Item $_ $MoveDirDouble ; "Moved Item $_" | Logging}
							$DoubleCountMoved++
						}
						else
						{
							"Move Failed as one or more files exist in destination path" | Logging "Red"
							$DoubleCountFailed++
						}
					}
				}
			}
		
		}
		
		$CountArray.DoubleCountMoved = $DoubleCountMoved
		$CountArray.DoubleCountFailed = $DoubleCountFailed
		
		#Loop through Singles
		if ($FolderGroupingSingle.Length -gt 0)
		{
		
		$SingleCountMoved = 0
		$SingleCountFailed = 0
		
			foreach ($GroupingSingle in $FolderGroupingSingle)
			{
				$GroupingSingle.Group | Logging "Yellow"
				
				if ($ReadOnly -eq 0)
				{
			
					$ErrorSingle = 0
					
					$GroupingSingleFile = $GroupingSingle.Group[0]
					
					$MoveFilePathSingle = $MoveDirSingle + "\" + $GroupingSingleFile.Name
					
					if (Test-Path $MoveFilePathSingle)
					{
						$ErrorSingle = 1
						"Move Failed as one or more files exist in destination path" | Logging "Red"
						$SingleCountFailed++
					}
					else
					{
						$GroupingSingle.Group | % {Move-Item $_ $MoveDirSingle ; "Moved Item $_" | Logging}
						$SingleCountMoved++
					}
					
				}
			}
		}
		
		$CountArray.SingleCountMoved = $SingleCountMoved
		$CountArray.SingleCountFailed = $SingleCountFailed
	
	}
	else
	{
		"No items found that match the extensions provided" | Logging "Red"
		break
	}
	
	"`n"
	"{0,-18} {1,0} {2,-2}" -f "TotalCount", ":", $CountArray.TotalCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCount", ":", $CountArray.SingleCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCountMoved", ":", $CountArray.SingleCountMoved | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "SingleCountFailed", ":", $CountArray.SingleCountFailed | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCount", ":", $CountArray.DoubleCount | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCountMoved", ":", $CountArray.DoubleCountMoved | Logging "Magenta"
	"{0,-18} {1,0} {2,-2}" -f "DoubleCountFailed", ":", $CountArray.DoubleCountFailed | Logging "Magenta"
	"`n"
	
}

Open in new window

0
 
LVL 11

Expert Comment

by:paultomasi
ID: 36996321
Mark...

I'd like to think you'd be excited.

I'm astounded it's taken this long and still no solution....
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36996755
I think the goal posts did move a few times to be fair!!  Bill and I did come up with some Batch alternatives back in http:#36975681 then http:#36979245 etc. but we've deviated from that spec. again and I haven't got time to rejig it now.
0
 

Author Comment

by:mholbert
ID: 36997097
Agreed.  What starts simple rarely stays that way.  

Seriously, i am quite impressed with how each of you jumped on this and supplied alternatives.  

Mark
0
 

Author Comment

by:mholbert
ID: 36997114
Chris.  the input is a little long, but not a problem as of yet.  I will keep playing with it and let you know if i run into issues, which i'm sure i will considering the playing field and my lack of playing time.

Mark
0
 

Author Closing Comment

by:mholbert
ID: 36997129
wow.  impressive response to what started as a thought and evolved into something quite helpful.  I appreciate the assistance.

Mark
0
 
LVL 10

Expert Comment

by:ReneGe
ID: 36997136
Mark,

It's ok to create multiple questions with a project. All are witnesses. That was my way of doing things. I asked a question, got an acceptable answer, then, leading me to another question. All clear and focused on a result. I learned so mutch that way.

I am following this question from the beginning hoping I could help, and must admit, it's focus got me somehow lost.

This post will be most likely be my contribution.

Cheers to you and all.
Rene
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36997932
lyes.  also doing it across multiple related q's does mean you get your original problem solved -- for instance by the three scripts provided at that point -- and then if you want to go off on a tangent take it into a new Q.  I followed your tangent first into going from a script that needed to prompt the user and count records into one that ran on a schedule for the pairs, or manually but frankly you lost me then when you went off with other requirements while sleeping here.

Just saying for another time.... If you keep shifitng focus then you end up losing people knowing what is going on, or giving up.

Steve
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 36997947
oh yeah.... Having written working code for one of your earlier paths we would then have some recognition at least ... As it stand three people here wrote you quite lengthy and detailed code and tested it without any recognition.
0
 

Author Comment

by:mholbert
ID: 37000174
Steve, excellent point.  I have been pondering exactly that and was planning on asking the question today.

How do I address the situation where I ask a question and get several responses?  In this case once I wrapped my brain around the situation the requirements morphed considerably, and that caused as much confusion on my end as it did on yours I suspect.  I originally tried Chris’s script, got errors and aborted because of my own copy and paste ineptitude apparently.  Then started with Bill’s script, then played with yours and was going to use it until Chris added a comment that I should really give PowerShell a shot, so I did and ended up back at the beginning.  All the scripts worked and would have solved the problem for me.

To add to the intrigue and to emphasize my gratitude for all of your assistance, I’m going to fill in the story a little.

I am bringing an Electronic Medical Records package live in a 50+ provider outpatient clinic and due to an unexpected last minute delay, ended up with 40k+ transcribed documents that had to be shoved into the database at the last minute.  The previous million files were scrubbed and cleaned over the last year to prepare them for the insertion, and were inserted about a month back, which should have been immediately prior to Go Live but due to the last minute delay, I ended up with 2 months files to address.

The files have to go through a conversion process for insertion into the database and are being read as pdf files and parsed for identifying information so they can be delivered into the database with no human interaction.  The problem I ran into was that while we cleaned the million files, the instructions to the transcriptionists to stop doing the things that caused it to take a year to clean the million never got passed and the files I ended up with were as messed up as the million were.  

So I started shoving the file through the process and they started failing at a number of levels.  The biggest problem I ran into was that the singles kept getting stuck in the input structure and the pairs needed to move.  That’s where this script question came from.  I was manually addressing the issue and realized I needed some automation to get me where I was going in the time allowed.  

But wait, there’s more.  As if the fun of an EMR implementation isn’t enough, my wife is going through treatment for inflammatory breast cancer and I am trying to do triple duty as IT Director, Daddy, Husband, etc.

So in the end, I’m getting my ass kicked on numerous fronts and one of the best decisions I have made in the recent past was to post the question.  I cannot adequately express my gratitude to each of you for the time spent and energy focused on a resolution to a total strangers question with nothing expected but the satisfaction of solving a problem, and some points.
Back to the original question.  How do I best address the situation where a question gets so many responses one simply has to be chosen.  Should I give everyone equal points for being willing to solve the problem, or give Chris more because he actually worked through to the end and give less points to everyone else?  That seems both fair and unfair because each of you was clearly willing to play all the way through it.

Sorry for the long response but I have been pondering this for several days and clearly need guidance on the appropriate way to proceed.

With Gratitude Overflowing,

Mark
0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 37004100
It's really your call how you close the question and allocate points.  If you end up moving forward with one approach or posted solution, then typically that is the one you would accept.  Then, many posters will often use the "assisted" option to select one or more post from any other experts that you feel added value to the discussion.

For most experts I don't think the absolute number of points is really the concern, but getting some recognition and appreciation for contributions helps them feel like their time was appreciated.  Some posters will just split the total points evenly among the accepted and assisted solutions, and in other cases they will give a little more to the accepted solution since that was the one that was most useful.  It's really up to you, and can vary based on the discussion for a particular question.

Hope this helps some,
~bp
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 37004719
yes that is about right bill.... The thing is we do this for free in spare time and in one question you may have a quick easy question and easy answer that takes you 10 secs to type in an answer.  in yours three of us have spent frankly far too long writing you lengthy scripts... An testing them.

Now what annoyed me here was just that as the goal posts moved earlier fully working, tetsed scripts became obsolete.

Dont need to do anything with this one but just suggesting for another time if things go off on a tangent it is probably posting  a related question, post a link in the old q and continue.  then those that solved your original request for instance wouldnt feel left out.

PS i know what it is like having 3 young boys, a medical conditions myself, though thankfully not an ill wife, business to run and what seems like a constant stream of school stuff - occasionally I fit in doing some paid work!  Anyway hope things settle down for you too soon.
0
 

Author Comment

by:mholbert
ID: 37005636
Steve, thanks for the kind thoughts and advice, i will definately keep my questions more focused in the future.  i am unclear where i look at the question to see who has gotten points and who hasn't and where do i hit the final button that says allocate all the points and mark the question resolved.

And the last follow up question.  you guys are impressive.  if i want to pay one/some of you to build for me, how do i go about that?

Mark
0
 
LVL 59

Expert Comment

by:Bill Prew
ID: 37007178
There are some guidelines in the Help area, in the FAQ.  Here's one example, but you may want to poke around a bit.

http://www.experts-exchange.com/help.jsp?hi=400#hs=8&hi=100

As far as professional work, I think many/most of the Experts that offer this type of service will provide some guidance in their profile (just click on an Experts name/id in the thread above and you will be taken there) on how to contact them if they are open to consulting for a fee.  That being said, even if they don't you could reach out here in a question as well.  Clearly EE isn't trying to provide full business solutions, it's more a Q&A type service (even though some great code shows up). So I don't think it's "against the rules" to ask an Expert who has responded to a posted question if they are interested in working a much larger need that you have for a fee.  

Steve, would you agree?

~bp
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 37008955
absolutely.  i have learned loads from ee myself over the years and has helped me with subjects that arent my core.  i did have one little follow up job somewhile back but frankly no-one has fancied paying in the, don't know but what 5000 questions say?

Always open to paid work.... Is quite quiet here in the uk at the mo. For me, especially harder not being able to drive :-(  like bill says there is a 'hire me' button in most peoples profiles, or I certainly have mail and web addresses in there too.   Likewise anyone can respond to a request for help in a q too..

Good luck with it and if you need help from Bill, I or any of the other good people here just ask!
0
 

Author Comment

by:mholbert
ID: 37009883
Interesting. i have two things i would not hesitate to spend a little money on to accomplish appropriately.

1. finalize the last few pieces of the powershell script Chris started.
2. Address an odd MS Word issue involving converting documents from being printed onto preprinted letter to instead being layered after completion with an electronic letterhead and therefore becoming printable on plain paper.

i would have to insist on paying Chris for the completion of the Powershell script.  I am open to suggestions and paying whoever jumps on to address item #2.

I believe is paying people for what they do and at this point you have already done a good bit for no pay, so how bout i pony up and make it even or better.
0
 
LVL 9

Expert Comment

by:chrismerritt
ID: 37034814
mholbert, sorry for my late reply i've been busy the past few days!

Of course I would always be interested in doing paid work, though that is not the main reason I answer questions here.

If you want to discuss it any further with me please feel free to email me on n1ghtwish AT hotmail dot com.

Thanks!
0

Featured Post

Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My attempt to use PowerShell and other great resources found online to simplify the deployment of Office 365 ProPlus client components to any workstation that needs it, regardless of existing Office components that may be needing attention.
In real business world data are crucial and sometimes data are shared among different information systems. Hence, an agreeable file transfer protocol need to be established.
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to go…
Starting up a Project

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question