?
Solved

How to split files in one directory into 2 or more directories?

Posted on 2015-01-09
5
Medium Priority
?
276 Views
Last Modified: 2015-01-12
Hi all!

I have some directories with thousands of files and I need to split those files into directories of 4,000 files or less. I'm fairly new to scripting so please bear with me. I've searched for a solution and I see that you can copy ALL files from a directory to a new directory but I want to copy only some of the files. All the files are the same type.

#split files into different directories if directory has more than 4,000 files

if $totalNumOfFiles > 4000

#Determine how many new directories are needed
numOfNewDir = int($totalNumOfFiles/4000)


#The new directories will be like this: original_folder_name_1, original_folder_name_2, etc.
#depending on how many new directories have to be created

sourceDirName = $DIR

while [ $numOfNewDir > 0 ]
do

destinationDirName = $sourceDirName_$numOfNewDir
mkdir $destinationDirName

#this is where I would split the files and copy them into a new directory but I don't know how to do it
#start copying 4,000 files

#cp $sourceDirName/files to $destinationDirName/files

#end copying, the next iteration would copy the next set of files starting at file number 4001

--numOfNewDir
done


done
0
Comment
Question by:afrocelt
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 40541540
sourceDirName=$DIR
files=( $sourceDirName/* )
if (( ${#files[*]} > 4000 )) ; then
  for (( i = 0 ; i < ${#files[@]} ; i++ )) do
    destinationDirName=${sourceDirName}_$((1+$i/4000))
    if [[ ! -d $destinationDirName ]] ; then  mkdir $destinationDirName ; fi
    mv "${files[$i]}" "$destinationDirName/"
  done
fi
0
 

Author Comment

by:afrocelt
ID: 40545348
Hi Ozo,

Thank you so much for the quick reply. I almost have it working except when the path contains spaces. I know I need to put quotes somewhere along here:

destinationDirName=${sourceDirName}_$((1+$i/4000))
f [[ ! -d $destinationDirName ]] ; then  mkdir $destinationDirName ;

because it tries multiple times to create a directory where there is a space:

If the directory is:  /Volumes/DirA/DirB/Personal Folders/me/THE FILES/image1.JPG

I get this:
mkdir: /Volumes/DirA/DirB/Personal: File exists
mkdir: Folders/me: No such file or directory
mkdir: FILES_5: File exists

Strangely, it still creates the correct path:

/Volumes/DirA/DirB/Personal Folders/me/THE FILES_5/

Do I need to put quotes around ${sourceDirName}? I tried a few iterations but I think I'm doing something wrong. Thanks in advance.
0
 
LVL 84

Accepted Solution

by:
ozo earned 2000 total points
ID: 40545355
files=( "$sourceDirName"/* )
if (( ${#files[*]} > 4000 )) ; then
  for (( i = 0 ; i < ${#files[@]} ; i++ )) do
    destinationDirName=${sourceDirName}_$((1+$i/4000))
    if [[ ! -d "$destinationDirName" ]] ; then  mkdir "$destinationDirName" ; fi
    mv "${files[$i]}" "$destinationDirName/"
  done
fi
0
 

Author Comment

by:afrocelt
ID: 40545393
Yes! That worked. I really appreciate your help.
0
 

Author Closing Comment

by:afrocelt
ID: 40545396
The response was quick as well as the answer to my follow-up question. Really helpful!
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Suggested Courses
Course of the Month14 days, 3 hours left to enroll

801 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question