Guideline for zipped & non zip files to be excluded from scanning

Trend's Deep Security has given us list of files (DB, certain Sharepoint files etc) to be excluded from
their AV scan as they will cause issue.

Trend's support has advised that zip files (esp those with many files zipped into one zip) will take
more resources to be unzipped, scanned & was told Deep Security will rezip it back.  They advised
that malware/viruses usually tend to infect smaller files & rarely infect big files but did not give
specific sizing of the files & the size of zip & non-zip files that will trigger systems performance
impact.

Anyone / any other AV products has any white paper / guidelines on
a) what's the file size above which malware generally won't infect
b) at what sizings (& number of files in a zip) that it's recommended
    not to scan so as not to affect performance.  In particular I have
    customer/tenants that use our facilities & publish zip & non-zip
    files & I would say it's fair if it takes 30 seconds to scan a published
    file, above which, the user will get unhappy

We run DS on-demand and realtime scan in Windows 2008 R2,
RHEL 5.x/6.x (realtime only) & Solaris x86 (on-demand)
sunhuxAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

MereteCommented:
Hi, personal opinion I don't think this is possible to set since malware constantly evolve/recode their scripts hence the reason we have updates to our our virus detection definitions,
if a weakness can be found they'll find it no matter the size.
a) what's the file size above which malware generally won't infect<< no such thing
b) at what sizings (& number of files in a zip) that it's recommended not scan, well Trend recommends for Compressed File Handling see Compressed files scanning restrictions:
btanExec ConsultantCommented:
I don't suggest we go by file size to scan but for performance sake based on AV, i do consider that their default is the best practice specific to their processing. There is no so called best practice as it is specific to the AV used.

E.g. in Symantec it has the Max File Size is 2 GB per File and if the File is a Container-File like ZIP etc. the unzipped size (including zip in zip) must not exeed 30 GB (30719MB).

But really do we even have process Files with Size up to 5 MB which is already consider quite huge already. I know the email attachment can go bigger than this but not often we do that and like need file transfer etc. I say the AV should not have limit though but earlier mentioned - take there recommended or default and tune as needed to your h/w and throughput. I do not see magic number appearing immediately but minimally you has the granularity to customise the sizing (this actually also complicate administrative task, go simple and have the biggest consumer in organisation to test out on their norm biggest file load then.

Nonetheless, you can check out the practice guide from TM

1 -(pdf) DSR page 34, it stated the recommended action and setting.
http://files.trendmicro.com/documentation/guides/deep_security/DS%209.0%20Best%20Practice%20Guide.pdf
e.g.
(Recommended Real-time Scan Configuration )
Maximum size of individual extracted files - 30
Maximum Levels - 2
Maximum number of files to extract -10
OLE Layers to Scan - 3

(Recommended Scheduled Scan Configuration )
Maximum size of individual extracted files - 60
Maximum Levels - 3
Maximum number of files to extract -10
OLE Layers to Scan - 3

(Recommended Manual Scan Configuration )
Maximum size of individual extracted files - 60
Maximum Levels - 2
Maximum number of files to extract -10
OLE Layers to Scan - 3

2-  (pdf) http://docs.trendmicro.com/all/ent/pp/v2.1/en-us/pp_2.1_bpg.pdf 
- See the "Advanced Options (Scan Restriction Criteria)" for
Decompressed file count exceeds ( default value is 9999), Size of decompressed file exceeds (default value is 100MB), Number of layers of compression exceeds (default value is five (5)), Size of decompressed file is “x” times the size of compressed file (default value is 1000)

Also do note that scanning a compressed file that might cause a Denial-of-Service (DoS) attack. A Denial-of-Service (DoS) attack happens when a mail server’s resources are overwhelmed by unnecessary tasks. So it is wise to also prevent AV from scanning files that decompress into very large files helps prevent this problem from happening. Hence those field do play a part  - just need to profile and tune as you go along ...

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:
This is quite a 'tricky' requirement: as the users published their files
& will need to view the outputs of the scan results on-the-fly, I would
say 20-30 secs of scanning is the max they can wait before they got
frustrated.

So be it zip or non-zip files, I'm trying to work out what is this 'magic'
figure of the file sizing, above which the scan will take more than 20
secs
Cloud as a Security Delivery Platform for MSSPs

Every Managed Security Service Provider (MSSP) needs a platform to deliver effective and efficient security-as-a-service to their customers. Scale, elasticity and profitability are a few of the many features that a Cloud platform offers. View our on-demand webinar to learn more!

btanExec ConsultantCommented:
indeed tricky but I do not think there is a magic no unless you try it out with baseline using the same spec as user machine. You have to do it for manual scan since user trigger the scan. For a start, try the recommended,

- if the time meet the 20-30sec then I suggest you go size 10% lower for buffering as there can be background process eating resource - it is minimal impact though.
- if the time is beyond 20-30sec, go for 50% lower and do another baselining and likewise if it is within the acceptable limit then have another 10% size lower.
Unless the support can suggest some perform figure which normally they will not as they will say environment specific and many dependency. So you likely have to do it yourself.

there is some old paper on DS comparison with some performance spec but timing of scan is not stated though (see "on demand scan"). it stated some file size prepopulated, for 50VM and took 14hr 16min
http://www.symantec.com/content/en/us/enterprise/other_resources/b-tolly-212117-sep12dot1-vmware-av-performance.pdf

best practice guide for DS 9
http://files.trendmicro.com/documentation/guides/deep_security/DS%209.0%20Best%20Practice%20Guide.pdf
sunhuxAuthor Commented:
Just thought of MS Excel (& a recent Powerpoint vulnerability) that may
have malicious macros : Ok, the largest Excel/Ppt file I've seen is 20MB.

We may have users uploading Excel to Sharepoint servers/VMs, so I
guess 20 MB is a decent magical number
btanExec ConsultantCommented:
yap test it as form of profiling and I should see this magic no isnt going to pose any issue to the TM DS unless support advice otherwise
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Anti-Virus Apps

From novice to tech pro — start learning today.