IBM Virtual Tape Appliance Research

We have an IBM as400 system for our primary EHR, and we're still backing it up with a physical tape drive. We're up to 3 tapes over a nine hour period of full downtime, and staff is upset about this much downtime every month.

My question is: does anyone have experience with virtual tape in comparison to physical tape? How much faster does it operate? IF I'm running the GO SAVE command option 21 on the as400 and it takes 9 hours, will virtual tape help me get it under 5 hours?

ANY and all feedback will be appreciated.
JHMH IT StaffAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Data sent to virtual tape drives go to disk, so many people think operations should run faster,
but you'll have to take into account that there's the need to emulate the behaviour of a tape device causing some overhead.
Besides that, modern "real" tape drives are not as slow as one should think.

The  throughput of an Ultrium 5 LTO drive is round about 140 MB/sec native, and 280 MB/sec assuming 2:1 compression.
Ultrium 6 can even go beyond 380-400 MB per second (2:1 compression assumed).
This is more than many disks can offer.

The high-end IBM ProtecTIER TS7650G gateway offers just 150 MB/sec in "single stream" mode which is only a little bit more than the (uncompressed) speed of a physical Ultrium 5 (see above).

If I remember well "GO SAVE" cannot write to multiple drives in parallel, so "single stream" is the only option here - and you have just this one AS/400, right?

Conclusion: You will see a noticeable improvement only if, at the moment, you're using a rather old and slow tape device.

I think it would be better considering the purchase of (for example) an 5638-8202 LTO-5 drive or the like. Compare the throughput of this device (see above) with those ~ 150 MB/sec of an expensive VTL and you'll get an idea.

Just for completeness I must add that if there were many systems at your site (AS/400 or other) which must be backed up you can benefit from the parallel processing capabilities of a VTL.
Gary PattersonVP Technology / Senior Consultant Commented:
Can virtual tape be faster than tape backup?  In some cases.  But, as woolmilcporc points out above, the relative difference depends entirely on your hardware.  For example, if you have single old, slow, tape drive, and a bunch of fast disk arms with plenty of space, you could see an improvement in time.  But if you have a fast tape drive, and a relatively small number of slow disk arms, then the opposite may be true - and especially if you are now going to be reading and writing to/from the same ASP at the same time.

Easiest way to tell is to just run a small benchmark save to each media and time each one.

We have customers running 24x7x365 with little or no annual downtime for backups.  For example, large shops use hardware or software replication strategies to eliminate the need to take the primary business system down for backups.  If replication isn't an option, we've had great success implementing the excellent save-while-active capabilties built into the system. SWA allows for backups to happen (generally in off-peak times) without the need to shut down end user processing - or to limit the outage to a short checkpoint period.  In one case, we were able to reduce a nightly 6-7 hour backup outage to less than 15 minutes.

With a little planning and knowledge of your specific system and applications, you can get even full system saves down to a very short time period using SAVACT.

You said "monthly", so I assume this might be a full system save.  It is true that SAVSYS requires the system to be in a restricted state, buy only the base component of SAVSYS requires all users to be off the system:


Once this has completed, you can bring the system back up,and use SAVCFG and SAVSECDTA to get the rest of the data usually backed up by SAVSYS.  Then you would use SAVLIB and SAV commands with SAVACT specified to backup the remainder of the system while applications are active.

You can also look at faster tape technologies, like multiple tape backup strategies, and at hardware Virtual Tape Libraries.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Thomas RushCommented:
Before we can suggest a solution with confidence, we need to know: Which LTO tape drive are you backing up to?   At least to know the generation: LTO-4, LTO-5, LTO-6, maybe earlier?, or IBM's model number.

Then: How much data are you backing up?  The solution may be very different if it's 1TB vs. 10 TB. vs. 100GB.   Knowing how much data will help us know your average backup performance.  How much of the nine hours is actual backup time (that is, are the tapes swapped promptly, or is there a significant wait time in between when a tape is full and the next one is available)?  Also helpful: Knowing how big the data is uncompressed, and how big it is compressed on the tape.

Knowing your average backup performance helps us understand where the bottleneck might be.  If you're only getting 30MB/sec to an LTO-3 or above, moving to a faster tape drive won't get your backups done any faster (save, perhaps, for not having to switch tapes in the middle) -- but neither will backing up to some kind of disk or VTL.   The low performance with a tape drive capable of writing much faster indicates that it's some other component which is the backup.  That bottleneck is very often the source disks.  It could be a mis-configured network or slow network (not so common now  that most are on GbE).  It could be processes like encryption or deduplication running on the backup server.

I can suggest that you check to make sure you are not compressing the data in software (that is, on the AS400 or backup server.  If so, turn that off and instead use "hardware compression", letting the tape drive do it without any performance penalty.

Likewise, if you are performing encryption on the backup server (or AS400), see if there is a solution that let's you use hardware encryption.  This is normally done through the backup software; I don't know how it would work in your environment.  Hardware encryption on LTO-4 or above tape drives has effectively no performance penalty (maybe 1% at most), as compared to a potential significant performance hit if using SW encryption.

Identifying the actual performance bottleneck before randomly buying more hardware is critical.  If you can only get 40MB/second off your disks, upgrading to a billionMB/sec tape drive won't help, because the disks can't get you more than 40MB/second.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

JHMH IT StaffAuthor Commented:
Here is some more information for you:

System: Power 720
Tape Drive: LTO 4
Tapes: LTO4, 800/1600 GB
Running HMS/Medhost

We have been using the default GO SAVE, Option 21 - so I am unsure of the compression type used by default. We run through 2 tapes and part of a third. I am not personally experienced enough with the system to locate the default encryption type, and the report I run after the backup only tells me the number of objects saved - not the amount of data.

We have already contacted Medhost and they will be of ZERO assistance in altering the backup strategy.
JHMH IT StaffAuthor Commented:
Also, there is never more than 5 minutes between tapes being changed - just thought I should answer that inquiry as well.
Gary PattersonVP Technology / Senior Consultant Commented:
GO SAVE, OPTION 21 defaults to DTACPR(*DEV) COMPACT(*DEV) SAVxxx options.  This tells the save commands to use native device compression or compaction if available, and if not, to perform no compression.

You don't mention your specific tape drive model, but I assume we're talking about a 3580 since you are talking about manually changing tapes.  3580 supports hardware data compaction - so that should be what is happening unless someone has modified command defaults or the save program.
If you already have LTO4 and if this AS/400 is the only system which uses the tape drive you certainly won't profit much from buying a VTL, let alone the value-for-money ratio.
JHMH IT StaffAuthor Commented:
Yes, based on the information provided on this thread our only value from a VTL would be not swapping tapes. So our only viable options would be a faster drive/tape scenario or devising a "save while active" strategy.
All you said in your last comment is  quite correct, indeed!
Thomas RushCommented:
So let's say 2.5 tape's worth of capacity, approximately 2TB, maybe more if your data is getting any kind of compression.  For what it's worth, my customers doing general backups typically see about 1.3:1 compression if I had to pick a median number.    2TB backed up over 9 hours means you're getting about 220GB/hour to the tape,   LTO-4's native speed is 120MB/second, or about 430GB/hour.   This implies that you're only getting about half (no compression) to 2/3 (1.3:1 compression) of the data that the tape drive could handle.

Therefore, an upgrade to LTO-6 should fit all your data on one tape, but will not result in faster backups other than the 10 minutes you save swapping tapes (because you'll still be getting only 220GB/hour to the [faster] tape drive).

A VTL won't help, either, and may give you slower backups(!), if it can't write a single stream at 60MB/sec (or 80MB/sec if your data is 1.3:1 compressible, and even faster, if your data is more compressible -- and getting a device that can do single streams over 80MB/sec tends to get expensive!).

If you can use the SAVACT recommended above, that's probably your best tool for better backups.
If not that, there is one other option I can see as an option, and that option works if your data is relatively static -- that is, among your 2TB of data, much of it does not change from week to week.

The solution revolves around a technology called "Incremental Forever with Synthetic Full Backups".   It is implemented in IBM's Tivoli Storage Manager, but I don't know if it's an option for your environment.  It works like this:
- You do one full backup to disk
- After that you do only incremental backups (which should be pretty quick, if most data doesn't change; you can do the incremental backups daily if you'd like -- or more or less often)
- Periodically (often once a week, could be less often depending on your requirements) you tell TSM to perform a "synthetic full backup", it looks at the files you have stored on disk, and creates a 'synthetic full' to tape from all the historical files that it's collected from that full and those incrementals, and it puts on tape a backup that has exactly the same files you'd have if you had done a full instead of your last incremental.   Since you're not going back to the original system, there's no downtime for your print server (or whatever).

TSM is not cheap, and it is not simple to manage (from what I hear), but if it's available, it may be your answer to keeping the application up with minimal downtime for backups, and yet providing you at least as much functionality and protection as you have today.
JHMH IT StaffAuthor Commented:
As our data changes very rapidly - we're a 24 hour orthopedic hospital  - I doubt that would be a solution. Accounts are worked on frequently, and not only are documents scanned in but forms are automatically generated and archived as PDFs - so there's a lot of changes every day.
TSM is a very fine product, we use it for more than 15 years now and I've become a big fan of it (better: an addict).

But in your environment there will be several heavy drawbacks, besides the fact that data change very rapidly:

- There is no TSM server version available for i5/OS, so you will have to purchase and install a new machine (AIX/Linux/Solaris/HPUX/Windows) to run the TSM server on. This machine must have sufficient power to run a DB/2 database (required by TSM) along with the basic TSM server code.

- A TSM server is not very easy to install and maintain. Most companies need external consulting/support people to do it, at least at the beginning.

- The TSM client for AS/400 is not as comprehensive as the client for other OSes. Basically it's an API interface between the native AS/400 BRMS and TSM. The BRMS application client to Tivoli Storage Manager requires "Backup Recovery and Media Services for iSeries", "OS/400 (i5/OS) Media Storage Extensions Feature" and "Tivoli Storage Manager Application Programming Interface for IBM i" (the API interface mentioned above). All this must be purchased by you.

- But now the biggest drawback: Only user data can be backed up to the Tivoli Storage Manager server. User data is any object that can be saved to a save file, but not "Licence Internal Code" (LIC), OS/400 (i5/OS) operating system, security data, configuration data, IBM supplied libraries and licensed programs.
System data can only be saved to local media and that's exactly the same as what you're doing now.

- In order to save user data the above mentioned new server must be equipped with sufficient storage media (disk/tape) which will have to be purchased in addition to the new TSM server hardware.

- TSM itself is not cheap (not at all!).

- In my previous comment I've been talking about the value-for money ratio of a VTL. I think you can imagine now that this ratio would be one to several orders of magnitude worse when talking about TSM in your environment.

To avoid misunderstandings: TSM is really fine for companies like ours with lots and lots of servers (no or only few AS/400s) and lots of terabytes of data to be saved. Once you have a well-performing server, a powerful tape library and sufficient space for backup-to-disk the life of a backup/recovery admin can become very easy.
Unfortunately all these critera do not seem to be met in your environment.
Carlos IjalbaIT Systems DirectorCommented:
TSM is an option on AS/400, but using it as a tape device for BRMS (Backup and Recovery Media Services), another product from IBM, but specific of the AS/400.

Is the AS/400 a stand alone model? or is it virtualized on a chassis? because you can create VTL on a VIOS using SSD disk to make it fly, however this is only good if you don't have too much data to back up, otherwise SSD is rather expensive.

You can consider buying a somewhat cheap TS3100 with 2 LTO6 tape drives, and run 2 backup processes nightly with SWA *yes. I think that the most important thing you've got to do with your backup policy is to identify the LIBs that your application uses and do daily backups, weekly, and only once a month do a complete system save (go save, 21), because performing a full system save more often seems overkill, specially when taking that long.

Other option will be a second AS/400 BDU to save licensing costs with replication soft like iCluster, VISION, etc, that way you can backup from the STDBY system, without impacting users, and having your system protected by HA.
JHMH IT StaffAuthor Commented:
The as400 is standalone, with 1 tape drive. A two system setup for replication is not economically feasible for our facility, so right now our best option is to develop a save-while-active strategy to minimize downtime.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Software

From novice to tech pro — start learning today.