TSM Scratch tape prediction

TSM will happily use a new scratch tape for each node "consuming" them all. using D2D2T and collocation groups and expiration smartly will greatly reduce number of almost empty media "consumed" merely recycling tapes for reuse.

Generally you will make at least 2 collocation groups - for archives and backups
Backups best stay in automated libray for quick access (reducing expiry period if no budget for now), (old)archive tapes can be filled and taken away as needed.

As previous expert mentioned - from 2 yearly consumptions you can not make any predictions since tivoli's consumption is F+I(x) - you need 3 reference points to make any predictions.

Thanks for the query.

I ran that query,  but for 2010 and 2011, I am getting zero output. For 2012 I am getting 37. Whether it means that volhistory table dont have data for 2010 and 2011. I know it is difficult to predict future tape growth. Is there a way to find out based on reclaimation,retention or something, whether my tape usage is getting wasted and atleast whether any tapes can be reclaimed asap to increase my scratch pool numbers?

virgo

VOLHISTORY normally lasts forever, unless you use to run DEL VOLH regularly wich you should avoid.

Of course I mean other commands than "DEL VOLH T=DBB ..." which every non-DRM user should run from time to time to free up DB backup volumes.

Do you clear DB backup volumes?  These volumes are never deleted automatically by TSM if you don't use DRM.

Issue "Q VOLH T=DBB". Lots of volumes? If so, and if you don't use DRM get rid of them:

DEL VOLH TOD=-10 T=DBB

This will delete all TSM DB backup tapes except for those created during the last 10 days (adjust this value if you want to keep more or less tapes).


Now, how old is your volume history?

Issue "Q VOLH T=STGNEW" and look at the timestamp of the first displayed entry - it shows how long your history actually traces back.

Another thing - if I remember well you're running TSM 5. The DB in that version is not as flexible as DB2 (TSM 6) in regard to timestamps.

Issue "select current date from db" - this will show you the standard date format which you should use in the query. So for example if you see "2012-03-20" use this format instead of the "03/20/2012" format I gave you.

Wasted tapes?

First of all, don't use COLLOCATE=NODE or (even worse) COLLOCATE=FILESPACE in the storage pool definitions if you can avoid it - these options are big tape consumers.

Don't run too many jobs against the same tape pool in parallel - this leads to more poorly utilized tapes.

Check how many underutilized tapes you have with this:

          10     select -
           20     substr(VOLUME_NAME,1,8) Name,  STGPOOL_NAME, PCT_UTILIZED -
           30     from VOLUMES -
           40     where -
           50     DEVCLASS_NAME='TAPECLASS' -
           55     and -
           60     PCT_UTILIZED<$1 -
           70     and -
           75     STATUS like 'FILLING' -
           80     order by 3 asc

Select allOpen in new window

Run this script e.g. with

RUN scriptname 3

This will show you all "FILLING" tapes with a uitilization lower than 3%.
(Again I used "TAPECLASS" for the tape device class name - please adjust.)

If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Have fun with TSM!

If you collocate by group you really need a disk pool where node data goes first, after staging/migration will orderly write data on 1 tape per collocation group.

Q STG F=D # will report media in a pool.

If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Select allOpen in new window

I ran that query and found that many tapes in one of my copy tape pool is underutilized. So, can I perform the move data on copy tape pool just like we do it on primary tape pool or whether there is a seperate procedure for the same. I am attaching the file for the reference containing output of query. Let me know what can be done in this case.

virgo

You can move files from a copy storage pool volume to volumes in the same copy storage pool, that's no problem.

But why should the volumes just in this copy pool be poorly utilized?
Does this storage pool differ in regard to the COLLOCATION setting?

Btw, seems you forgot to attach the output file.

this time with the file. Also, collocation details for the storage pool is as follows :


STGPOOL_NAME           COLLOCATE              NUMSCRATCHUSED
------------------     ------------------     --------------
CBACK6                 NO                                254
TBACK6                 NO                                241
CBACK5                 NO                                137
TBACK5                 NO                                120
CBACK2                 NO                                113
CBACK1                 NO                                101
TBACK2                 NO                                101
CBACK3                 NO                                 19
TBACK1                 NO                                 15
TBACK33                NODE                               15
TBACK11                NO                                  5


I dont see collocation is on for cback1, but still it is showing lots of underutilized tapes.
util-query.txt

Strange.

Is CBACK1 the backup target for many primary pools, and do you run the BACKUP STGPOOL jobs in parallel, with a huge MAXPROCESS setting?

How can I check for which primary pool CBACK1 is the target pool?

And also when I see my daily DRM script, the backup stg jobs are running with maxprocess of 4.

>> How can I check for which primary pool CBACK1 is the target pool? <<

That's specified in your backup stgp jobs, nowhere else.

4 processes doesnt's sound too exaggerated, unless you use to run many jobs in parallel.
If you have indeed many such jobs consider specifying WAIT=YES.

Collocation doesn't seem to be the culprit, because there's just one pool with collocation enabled, and this one has just 15 volumes.

I'd suggest as a first step reducing the number of volumes in CBACK1 by means of MOVE DATA to then watch carefully whether this number would again grow enormously,

Is it ok to directly run "move data vol" on the cback1 underutilized volumes. Whether it will create any problem for the volumes which are showing as offsite or unavailable?

Yes, cou can move files from an offsite volume in a copy storage pool.
Because the offsite volume cannot be mounted, the TSM server obtains the files
that are on the offsite volume from the primary storage pool (or perhaps from another copy storage pool, if there is one containing the data).
These files are then written to the destination volumes in the original copy storage pool
and the "offsite" volume will become logically empty that way.

Unavailable volumes cannot be the source of MOVE DATA, I think.

I see there are total 78 volumes in cback1 which are utilized less that 3%, but all of them Access mode is showing as offsite. So if I start move data for them the data will be moved to other tapes in the same pool and whether it will mark that tape as scratch/empty.

If it is marking it as empty, then whether I can recall them immediately from Iron mountain and use them as a scratch tapes?

Also, move data will move the data to the other tapes in the cback1 pool, so in that case how my data will be maintained at offsite?

Yes, the tapes will be empty once MOVE DATA has completed.

A MOVE DRMEDIA TOSTATE=ONSITER operation will bring them back onsite immediately.

Data which have been subject to MOVE DATA will no longer be offsite.
The next MOVE DRMEDIA TOSTATE=VAULT operation will bring them (their volumes, that is) offsite again.

we have a script running at 4.15 pm daily which requests for empty tapes from onsite. In this script the command is :

q drmedia wherestate=vaultr

whether using this command it will recall that empty tape. I see that tape when I execute the command in TSM. So my question here is, if I create a script to move data and schedule it so that it will move data from under utilized tapes and then mark them as empty so when tomorrow my tape request script runs,  it should recall all that tapes ... right?

"Q" DRMEDIA will obviously not move any tapes, but yes, it will display the now empty tapes as "vault retrieveable".

A subsequent "MOVE DRMEDIA TOSTATE=ONSITER"  will bring the tapes back then.

ok, got it and thanks for all that help and guidance for this issue.

virgo

wmp,

Ok, so i was going through my DRM script and found that there is following entry which deletes the volume history older that 32 days.

DEL VOLHIST TODATE=TODAY-32 type=all

 If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. can you confirm the same. Also what is "type=all" here?

virgo

volume type - scratch/online/offsite  etc

thanks gheist...? but I need answer on my other part of the question also.

>> If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. <<

Yes, but you'll have to wait until a history spanning the desired time range has accumulated, deleted volhistory records will of course never come back.

TYPE=ALL means deleting history information about:

- Backupsets
- DB Backups
- DB Snapshots
- DB Rpf (= DB backups + Recovery plans)
- Export Volumes
- Library Client (remote) Volumes
- RP Files, RP Snapshots (Recovery plans)
- STGNEW (volumes added to stgpools e.g. from the scratch pool)
- STGDELETE (volumes returned to the scratch pool or deleted)
- STGREUSE (empty volumes being reused, valid for stgpools which are not allowed to use scratch volumes but have fixed volumes instead).

Thanks for the explanation. I will verify my configuration and accordingly configure the volume history.

Thanks
virgo

I am not using collocation on my storage pools, infact I disabled it one on of my storage pools due to which there were so many underutilized tapes, on which I performed move data and now I am using them for scratch. I freed up nearly 80-90 tapes.

For future tape growth forecasting, I am looking at Servergraph to see if there are any options or reports that can be configured.

virgo

If you switch off collocation you lose ability to restore fast. Having same node on set of tapes less than you have drives makes restore optimal. I think MMC reporting component has some reporting configuration (external to TSM) which emits global report and per-node reports to node managers (it takes time to get it right, look at locally configured mail relay and ask your mail admin for more info, it will merely send a mail per day per node)

PS that is why i recommend the per-node approach, at least starting with services facing customers and services facing whole enterprise.
once there is downtime you do not have much breathing space to improve architecture under hood.