TSM Scratch tape prediction

Hi All,

I want to predict my TSM scratch tape usage for the next one year for budgetary reasons. How it can be done so that I can do the capacity planning for the tapes I need to purchase. Also, is there any query where in I can find out the comparison of last year and this year scratch tape usage?

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

There is obviously no chance to predict your scratch tape requirements.

How should TSM know about the (positive or negative) growth of the amount of data to be backed up?

I can give you a query which will try to determine the scratch tape consumption in the past (2010-2012) on the basis of the VOLHISTORY table, by subtracting the number of STGDELETE operations (release reclaimed tape) from the number of STGNEW operations (add scratch tape to pool)..

To restrict the query to TAPE operations (as opposed to FILE operations) we need the name(s) of your tape DEVCLASS(ES).
I assume "TAPECLASS" here.
               1          select -                                                    
               6           ( select count(*) from VOLHISTORY where -                  
               11             TYPE='STGNEW' and -                                     
               16             DEVCLASS='TAPECLASS' and -                             
               21             DATE_TIME between '01/01/2010' and '12/31/2010' ) -     
               26         - ( select count(*) from VOLHISTORY where -                 
               31              TYPE='STGDELETE' and -                                 
               36              DEVCLASS='TAPECLASS' and -                            
               41              DATE_TIME between '01/01/2010' and '12/31/2010' ) -    
               46               as "SCR 2010" from DB                            
               51         select -                                                    
               56          ( select count(*) from VOLHISTORY where -                  
               61             TYPE='STGNEW' and -                                     
               66             DEVCLASS='TAPECLASS' and -                             
               71             DATE_TIME between '01/01/2011' and '12/31/2011' ) -     
               76         - ( select count(*) from VOLHISTORY where -                 
               81              TYPE='STGDELETE' and -                                 
               86              DEVCLASS='TAPECLASS' and -                            
               91              DATE_TIME between '01/01/2011' and '12/31/2011' ) -    
               96               as "SCR 2011" from DB                            
               101        select -                                                    
               106         ( select count(*) from VOLHISTORY where -                  
               111            TYPE='STGNEW' and -                                     
               116            DEVCLASS='TAPECLASS' and -                             
               121            DATE_TIME between '01/01/2012' and '12/31/2012' ) -     
               126        - ( select count(*) from VOLHISTORY where -                 
               131             TYPE='STGDELETE' and -                                 
               136             DEVCLASS='TAPECLASS' and -                            
               141             DATE_TIME between '01/01/2012' and '12/31/2012' ) -    
               146        as "SCR 2012" from DB

Open in new window

I think I should repeat: There is no possibility to make any predictions. Please don't insist!

Maybe there's something in the GUI (yet I don't believe so), but since I never use the GUI I can't help you with this.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
TSM will happily use a new scratch tape for each node "consuming" them all. using D2D2T and collocation groups and expiration smartly will greatly reduce number of almost empty media "consumed" merely recycling tapes for reuse.

Generally you will make at least 2 collocation groups - for archives and backups
Backups best stay in automated libray for quick access (reducing expiry period if no budget for now), (old)archive tapes can be filled and taken away as needed.

As previous expert mentioned - from 2 yearly consumptions you can not make any predictions since tivoli's consumption is F+I(x) - you need 3 reference points to make any predictions.
virgo0880Author Commented:
Thanks for the query.

I ran that query,  but for 2010 and 2011, I am getting zero output. For 2012 I am getting 37. Whether it means that volhistory table dont have data for 2010 and 2011. I know it is difficult to predict future tape growth. Is there a way to find out based on reclaimation,retention or something, whether my tape usage is getting wasted and atleast whether any tapes can be reclaimed asap to increase my scratch pool numbers?

Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

VOLHISTORY normally lasts forever, unless you use to run DEL VOLH regularly wich you should avoid.

Of course I mean other commands than "DEL VOLH T=DBB ..." which every non-DRM user should run from time to time to free up DB backup volumes.

Do you clear DB backup volumes?  These volumes are never deleted automatically by TSM if you don't use DRM.

Issue "Q VOLH T=DBB". Lots of volumes? If so, and if you don't use DRM get rid of them:


This will delete all TSM DB backup tapes except for those created during the last 10 days (adjust this value if you want to keep more or less tapes).

Now, how old is your volume history?

Issue "Q VOLH T=STGNEW" and look at the timestamp of the first displayed entry - it shows how long your history actually traces back.

Another thing - if I remember well you're running TSM 5. The DB in that version is not as flexible as DB2 (TSM 6) in regard to timestamps.

Issue "select current date from db" - this will show you the standard date format which you should use in the query. So for example if you see "2012-03-20" use this format instead of the "03/20/2012" format I gave you.

Wasted tapes?

First of all, don't use COLLOCATE=NODE or (even worse) COLLOCATE=FILESPACE in the storage pool definitions if you can avoid it - these options are big tape consumers.

Don't run too many jobs against the same tape pool in parallel - this leads to more poorly utilized tapes.

Check how many underutilized tapes you have with this:
          10     select -
           20     substr(VOLUME_NAME,1,8) Name,  STGPOOL_NAME, PCT_UTILIZED -
           30     from VOLUMES -
           40     where -
           50     DEVCLASS_NAME='TAPECLASS' -
           55     and -
           60     PCT_UTILIZED<$1 -
           70     and -
           75     STATUS like 'FILLING' -
           80     order by 3 asc

Open in new window

Run this script e.g. with

RUN scriptname 3

This will show you all "FILLING" tapes with a uitilization lower than 3%.
(Again I used "TAPECLASS" for the tape device class name - please adjust.)

If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Have fun with TSM!
If you collocate by group you really need a disk pool where node data goes first, after staging/migration will orderly write data on 1 tape per collocation group.

Q STG F=D # will report media in a pool.
If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Open in new window

virgo0880Author Commented:
I ran that query and found that many tapes in one of my copy tape pool is underutilized. So, can I perform the move data on copy tape pool just like we do it on primary tape pool or whether there is a seperate procedure for the same. I am attaching the file for the reference containing output of query. Let me know what can be done in this case.

You can move files from a copy storage pool volume to volumes in the same copy storage pool, that's no problem.

But why should the volumes just in this copy pool be poorly utilized?
Does this storage pool differ in regard to the COLLOCATION setting?

Btw, seems you forgot to attach the output file.
virgo0880Author Commented:
this time with the file. Also, collocation details for the storage pool is as follows :

------------------     ------------------     --------------
CBACK6                 NO                                254
TBACK6                 NO                                241
CBACK5                 NO                                137
TBACK5                 NO                                120
CBACK2                 NO                                113
CBACK1                 NO                                101
TBACK2                 NO                                101
CBACK3                 NO                                 19
TBACK1                 NO                                 15
TBACK33                NODE                               15
TBACK11                NO                                  5

I dont see collocation is on for cback1, but still it is showing lots of underutilized tapes.

Is CBACK1 the backup target for many primary pools, and do you run the BACKUP STGPOOL jobs in parallel, with a huge MAXPROCESS setting?
The damage is done in the beginning when TSM allocates tape per node. you need to explicitly move data out from underutilised tapes once you get collocation groups right.
virgo0880Author Commented:
How can I check for which primary pool CBACK1 is the target pool?

And also when I see my daily DRM script, the backup stg jobs are running with maxprocess of 4.
>> How can I check for which primary pool CBACK1 is the target pool? <<

That's specified in your backup stgp jobs, nowhere else.

4 processes doesnt's sound too exaggerated, unless you use to run many jobs in parallel.
If you have indeed many such jobs consider specifying WAIT=YES.

Collocation doesn't seem to be the culprit, because there's just one pool with collocation enabled, and this one has just 15 volumes.

I'd suggest as a first step reducing the number of volumes in CBACK1 by means of MOVE DATA to then watch carefully whether this number would again grow enormously,
virgo0880Author Commented:
Is it ok to directly run "move data vol" on the cback1 underutilized volumes. Whether it will create any problem for the volumes which are showing as offsite or unavailable?
Yes, cou can move files from an offsite volume in a copy storage pool.
Because the offsite volume cannot be mounted, the TSM server obtains the files
that are on the offsite volume from the primary storage pool (or perhaps from another copy storage pool, if there is one containing the data).
These files are then written to the destination volumes in the original copy storage pool
and the "offsite" volume will become logically empty that way.

Unavailable volumes cannot be the source of MOVE DATA, I think.
virgo0880Author Commented:
I see there are total 78 volumes in cback1 which are utilized less that 3%, but all of them Access mode is showing as offsite. So if I start move data for them the data will be moved to other tapes in the same pool and whether it will mark that tape as scratch/empty.

If it is marking it as empty, then whether I can recall them immediately from Iron mountain and use them as a scratch tapes?

Also, move data will move the data to the other tapes in the cback1 pool, so in that case how my data will be maintained at offsite?
Yes, the tapes will be empty once MOVE DATA has completed.

A MOVE DRMEDIA TOSTATE=ONSITER operation will bring them back onsite immediately.

Data which have been subject to MOVE DATA will no longer be offsite.
The next MOVE DRMEDIA TOSTATE=VAULT operation will bring them (their volumes, that is) offsite again.
virgo0880Author Commented:
we have a script running at 4.15 pm daily which requests for empty tapes from onsite. In this script the command is :

q drmedia wherestate=vaultr

whether using this command it will recall that empty tape. I see that tape when I execute the command in TSM. So my question here is, if I create a script to move data and schedule it so that it will move data from under utilized tapes and then mark them as empty so when tomorrow my tape request script runs,  it should recall all that tapes ... right?
"Q" DRMEDIA will obviously not move any tapes, but yes, it will display the now empty tapes as "vault retrieveable".

A subsequent "MOVE DRMEDIA TOSTATE=ONSITER"  will bring the tapes back then.
virgo0880Author Commented:
ok, got it and thanks for all that help and guidance for this issue.

virgo0880Author Commented:

Ok, so i was going through my DRM script and found that there is following entry which deletes the volume history older that 32 days.


 If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. can you confirm the same. Also what is "type=all" here?

volume type - scratch/online/offsite  etc
virgo0880Author Commented:
thanks gheist...? but I need answer on my other part of the question also.
>> If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. <<

Yes, but you'll have to wait until a history spanning the desired time range has accumulated, deleted volhistory records will of course never come back.

TYPE=ALL means deleting history information about:

- Backupsets
- DB Backups
- DB Snapshots
- DB Rpf (= DB backups + Recovery plans)
- Export Volumes
- Library Client (remote) Volumes
- RP Files, RP Snapshots (Recovery plans)
- STGNEW (volumes added to stgpools e.g. from the scratch pool)
- STGDELETE (volumes returned to the scratch pool or deleted)
- STGREUSE (empty volumes being reused, valid for stgpools which are not allowed to use scratch volumes but have fixed volumes instead).
virgo0880Author Commented:
Thanks for the explanation. I will verify my configuration and accordingly configure the volume history.

You also need to eliminate underused volumes after you set more or less logical collocation groups. You can use "MOVE NODED" to move particular nodes in same collocation group away from current tapes, so that collocation applies, and do so until target volume reaches 50%....
virgo0880Author Commented:
I am not using collocation on my storage pools, infact I disabled it one on of my storage pools due to which there were so many underutilized tapes, on which I performed move data and now I am using them for scratch. I freed up nearly 80-90 tapes.

For future tape growth forecasting, I am looking at Servergraph to see if there are any options or reports that can be configured.

If you switch off collocation you lose ability to restore fast. Having same node on set of tapes less than you have drives makes restore optimal. I think MMC reporting component has some reporting configuration (external to TSM) which emits global report and per-node reports to node managers (it takes time to get it right, look at locally configured mail relay and ask your mail admin for more info, it will merely send a mail per day per node)
PS that is why i recommend the per-node approach, at least starting with services facing customers and services facing whole enterprise.
once there is downtime you do not have much breathing space to improve architecture under hood.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Software

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.