Backup Security post Ransomware Incident

1. Train users not open emails from strangers.
2. Use a top-notch anti-spam application to ditch over 90% of ransomware emails.
3. Use either an Azure cloud backup solution or a backup tape drive with backups rotated off site.

The source of the ransomware is most via email. Stop these emails.

John hit the nail on the head. one additional bit of advice is creating a good backup retention policy. my favorite policy has been the grandfather-father-son retention policy.  it takes everything into account if you retain the backups properly. had they been using this policy most or at the least some of their data could have been recovered.
it is basically this...

Son - keep all backups for 8 business days.  
Father - Keep weekly backups for 5 weeks
Grandfather - keep monthly backups for 3 months.

keep backups offline.
perform full backups if you have the time or full backups for weekly and monthly and Differential for the backups that run during the week. if using a tape drive and you can fit the full backup on one tape you will need to have 17 tapes in the rotation.

you would have had 16 different points in history to restore.  more than likely you would have had to only lose a few days of work.  it will also cover you for disaster, hardware failure, theft, and software corruption.

All good points that I agree with and will work with going forward.

Would you all say that I am focusing too much on granular detail rather than fundamentals? For example, is having that SON/FATHER/GRANDFATHER retention + offsite + offline backups WAY more important than HOW I attach a NAS to the server?

I do see a lot of good points that I will probably make separate posts to get advice on.

I think user education is the single biggest gap I need to deal with. Being a very small company they are not very open to "training".
Is there an affordable vendor for training, such as false spam that track who identifies it? Maybe some kind of reward system could keep their attention.

Training is the most important point and controlling spam the second most important point.

yes,  I used this product a few years ago. for the entire organization, a one-year subscription covering 60 mailboxes was around 1000$. you can create a monthly campaign that will send out spam.  it will track the spam and provide immediate feedback to the end user.  lost of reporting options.  great video training.  very nice product.  

Quote from their site. I think they are pretty proud of it.
We Will Pay Your Crypto-Ransom If You Get Hit With Ransomware

https://www.knowbe4.com/

one backup must always be offline ransomware may hit when both are online and you lose both. The 3-2-1 rule is there for a reason.. 3 copies, 2 media, 1 offsite

David I hear what you are saying.

Ransonware can also be dormant for a period of time. The more history you have offline the better. Considering the drastic drop in media costs why shouldn’t you have a years worth of month end backups? With the gf-f-s scenario I outlined above 7 more pieces of media offline would take you back a year.

I would like to point out that the gf-f-s retention policy has been around since I started in IT in the 90’s.  Back then it was in place to make sure you could restore data that was important but no one noticed it was deleted.  Something like an annual accounting report that was only used once a year. You could keep the grandfather for 12 months and have a year end that you kept for as long as the media would allow. Call it the great grandfather. Remember archival quality media?

With all the cloud storage options available today most people think that it’s enough to have a live copy in two places. Maybe one offline copy off-site just Incase.   I hate to admit that I still go to Best Buy the first of January every year and buy some sort of storage to back up every bit of data I own and squirrel it away.  in 20 years I have probably spent 3k in hard drives setting in a rubbermaid storage container. But I have gone back for pictures, PDFs of tax returns, documents I have written, copies of bills, invoices, online receipts, warranty documents you name it.

I have also lived through a natural disaster. I was working full time as an it manager and consulting for 6 other small businesses when it happened. All of them were doing online backup with deduplication to local storage. All of them were able to access payroll information and start communicating with their customers and clients within 48 hours and lost little or no data.

Online back for disaster recovery and offline for archive. It happens.
https://en.m.wikipedia.org/wiki/2011_Joplin_tornado

Training as once off will not be useful but suggest as a small company, focus more on user vigilance on "fakes" with high risk symptoms
a) Detecting phishing email (red flags - unknown sender, scam content, suspicious attachment and URL, etc )
b) Safe handling of USB drives (high risk - autorun type, unknown device picked up, gift from conference, positive findings on AV scan etc)
c) Use of Mobile device  and apps (advisory - do not install from unknown app store, do not jailbreak /root device, beware of apps asking for lots of permission etc)

KnowBE4 and PhishME are good candidate worth bringing the discussion further. Discipline on cyber hygiene is way forward to keep the security awareness on constant look out for "fake" surfacing in the internet...

The only two realistic preventions for ransomware is versioned controlled backup and whitelisted application access to document location.

>>Long story short - My client did not agree to an end to end backup solution and lost everything to ransomware.<<
In this case, the company could benefit from having an off-site backup of all data and systems.


You should have an effective Lesson Learned process that covers this incident, the following are just a few of the questions that you should ask when you build your Lessons Learned
• What actions did you take?
• Is this the optimal solution?
• Are there more capable solutions out there?
• How did the company react to the issue?
• In the event of the same or a similar incident occurring, how would you respond differently?
Based on above questions update the security policy


Also build procedures that contain the best practices listed as follows: (few points)

• Dictates that users or systems should only have the minimal level of access to perform the duties required of them. (Least privilege)
• Defines monitoring, response, and reporting requirements for incidents that involve security breaches.
• Security awareness and training both play a role in incident response, without comprehensive education, user-based attacks, such as social engineering, will be a major source of risk
• Investigate from where a breach emanated, how a breach might have occurred, and who might be responsible for the breach.

https://www.experts-exchange.com/articles/31763/Incident-Handling-and-Response-Plan.html

I asked a broad question so there is no one specific answer, that said, everyone has helped a lot in giving me food for thought. I will put some of this into writing up procedures, configurations, and training. Thank you all!

Yes, the best and the most important mitigation technique is to maintain secure backups and employee education. Educate employees - Just Say No — To Suspicious Emails and Links.
Use KnowBe4 to train users and email filter like Barracuda.
Use strong passwords.

Ways to Address Ransomware Attacks in Today’s Security Landscape:
https://www.lepide.com/blog/nine-ways-to-address-ransomware-attacks-in-todays-security-landscape/

Simple things you can do to protect against ransomware attacks:
http://expert-advice.org/security/ways-to-protect-yourself-from-ransomware-attack/