Avatar of dougp23
dougp23
Flag for United States of America asked on

Search many Zip Files

Our email system does a backup each day of all email sent and received, and stores it in a directory with the date stamp, so:

mail/2010-10-10/2010-10-10.zip
mail/2010-10-11/2010-10-11.zip

etc

I now have to provide all emails from jsmith@whatver.com from 2010-10-01 to 2010-12-31.  Of course, this is impossible to do until I unzip everything!

There has got to be a way to automate this!  I have tried Recoll, but I must be doing something wrong, every search returns nothing (even searching items I KNOW are there).

Appreciate any help!
Shell Scripting

Avatar of undefined
Last Comment
farzanj

8/22/2022 - Mon
farzanj

It can sure be scripted very easily.  But I would need more information.

First step is to unzip those files to a directory.
Then command to upzip all would look like this, may be not exactly


 
for m in {10..12}
do
    for d in $(seq -w 01 31)
   do
        date="2010-$m-$d"
        unzip -j mail/$date/$date.zip
    done
done

Open in new window


Then get one email and then the question would become, how to extract the emails you want from those files.
dougp23

ASKER
Your bash script is a little beyond me (as I fiddle with the variables,lol).  All the directories that I need to step through are in one top level dir (/mail).  

So could we say

for all directories in /mail
unzip the zip in each directory.

For the 2nd part of your question, each email is extracted with a unique time stamp for that day, so once unzipped, each dir will include every email sent and received that day.  I can then grep or egrep for a person's email address that I need.

Thanks!
farzanj

First part.
I would like to keep your zip files intact.  So, please put those to a new location, say test folder.  This way you would not regret.

Second part, I would need to see the file structure to construct a regular expression for extraction because, I am expecting emails to be in the following format

HEADERS
body

NEXT EMAIL HEADERS

I think you want to get the emails along with the body, so I need to setup a rule like
start from a line that contains this specific field and end with a line where you see this.
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
dougp23

ASKER
If I unzip one of the zips, I get files like this:

09:39:27-0400.12630.0
09:40:09-0400.12630.0
09:40:09-0400.12630.1

So as you can see, each email is self-contained.  If I do a cat or a less of the file, I can see all the headers as well as the body.  So I just need to walk through each directory and unzip.
farzanj

Ok, I would give you the code if you answer three questions.

In an email, is there a field in the header called from: jsmith@whatver.com something like this.  I want the exact syntax.

Second, should I follow exactly this directory structure?

Third, what should be the file or folder where I save it?
ASKER CERTIFIED SOLUTION
farzanj

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
farzanj

Sorry.

Change line 13 above to:
unzip mail/$date/$date.zip -d $TMP
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.