Solved

TSM Scratch tape prediction

Posted on 2012-03-19
28
2,462 Views
Last Modified: 2012-04-16
Hi All,

I want to predict my TSM scratch tape usage for the next one year for budgetary reasons. How it can be done so that I can do the capacity planning for the tapes I need to purchase. Also, is there any query where in I can find out the comparison of last year and this year scratch tape usage?

Thanks
Virgo
0
Comment
Question by:virgo0880
  • 12
  • 9
  • 7
28 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 400 total points
ID: 37741142
There is obviously no chance to predict your scratch tape requirements.

How should TSM know about the (positive or negative) growth of the amount of data to be backed up?

I can give you a query which will try to determine the scratch tape consumption in the past (2010-2012) on the basis of the VOLHISTORY table, by subtracting the number of STGDELETE operations (release reclaimed tape) from the number of STGNEW operations (add scratch tape to pool)..

To restrict the query to TAPE operations (as opposed to FILE operations) we need the name(s) of your tape DEVCLASS(ES).
I assume "TAPECLASS" here.
               1          select -                                                    
               6           ( select count(*) from VOLHISTORY where -                  
               11             TYPE='STGNEW' and -                                     
               16             DEVCLASS='TAPECLASS' and -                             
               21             DATE_TIME between '01/01/2010' and '12/31/2010' ) -     
               26         - ( select count(*) from VOLHISTORY where -                 
               31              TYPE='STGDELETE' and -                                 
               36              DEVCLASS='TAPECLASS' and -                            
               41              DATE_TIME between '01/01/2010' and '12/31/2010' ) -    
               46               as "SCR 2010" from DB                            
               51         select -                                                    
               56          ( select count(*) from VOLHISTORY where -                  
               61             TYPE='STGNEW' and -                                     
               66             DEVCLASS='TAPECLASS' and -                             
               71             DATE_TIME between '01/01/2011' and '12/31/2011' ) -     
               76         - ( select count(*) from VOLHISTORY where -                 
               81              TYPE='STGDELETE' and -                                 
               86              DEVCLASS='TAPECLASS' and -                            
               91              DATE_TIME between '01/01/2011' and '12/31/2011' ) -    
               96               as "SCR 2011" from DB                            
               101        select -                                                    
               106         ( select count(*) from VOLHISTORY where -                  
               111            TYPE='STGNEW' and -                                     
               116            DEVCLASS='TAPECLASS' and -                             
               121            DATE_TIME between '01/01/2012' and '12/31/2012' ) -     
               126        - ( select count(*) from VOLHISTORY where -                 
               131             TYPE='STGDELETE' and -                                 
               136             DEVCLASS='TAPECLASS' and -                            
               141             DATE_TIME between '01/01/2012' and '12/31/2012' ) -    
               146        as "SCR 2012" from DB

Open in new window

I think I should repeat: There is no possibility to make any predictions. Please don't insist!

Maybe there's something in the GUI (yet I don't believe so), but since I never use the GUI I can't help you with this.
0
 
LVL 61

Expert Comment

by:gheist
ID: 37741469
TSM will happily use a new scratch tape for each node "consuming" them all. using D2D2T and collocation groups and expiration smartly will greatly reduce number of almost empty media "consumed" merely recycling tapes for reuse.

Generally you will make at least 2 collocation groups - for archives and backups
Backups best stay in automated libray for quick access (reducing expiry period if no budget for now), (old)archive tapes can be filled and taken away as needed.

As previous expert mentioned - from 2 yearly consumptions you can not make any predictions since tivoli's consumption is F+I(x) - you need 3 reference points to make any predictions.
0
 

Author Comment

by:virgo0880
ID: 37743489
Thanks for the query.

I ran that query,  but for 2010 and 2011, I am getting zero output. For 2012 I am getting 37. Whether it means that volhistory table dont have data for 2010 and 2011. I know it is difficult to predict future tape growth. Is there a way to find out based on reclaimation,retention or something, whether my tape usage is getting wasted and atleast whether any tapes can be reclaimed asap to increase my scratch pool numbers?

virgo
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37743916
VOLHISTORY normally lasts forever, unless you use to run DEL VOLH regularly wich you should avoid.

Of course I mean other commands than "DEL VOLH T=DBB ..." which every non-DRM user should run from time to time to free up DB backup volumes.

Do you clear DB backup volumes?  These volumes are never deleted automatically by TSM if you don't use DRM.

Issue "Q VOLH T=DBB". Lots of volumes? If so, and if you don't use DRM get rid of them:

DEL VOLH TOD=-10 T=DBB

This will delete all TSM DB backup tapes except for those created during the last 10 days (adjust this value if you want to keep more or less tapes).


Now, how old is your volume history?

Issue "Q VOLH T=STGNEW" and look at the timestamp of the first displayed entry - it shows how long your history actually traces back.

Another thing - if I remember well you're running TSM 5. The DB in that version is not as flexible as DB2 (TSM 6) in regard to timestamps.

Issue "select current date from db" - this will show you the standard date format which you should use in the query. So for example if you see "2012-03-20" use this format instead of the "03/20/2012" format I gave you.

Wasted tapes?

First of all, don't use COLLOCATE=NODE or (even worse) COLLOCATE=FILESPACE in the storage pool definitions if you can avoid it - these options are big tape consumers.

Don't run too many jobs against the same tape pool in parallel - this leads to more poorly utilized tapes.

Check how many underutilized tapes you have with this:
          10     select -
           20     substr(VOLUME_NAME,1,8) Name,  STGPOOL_NAME, PCT_UTILIZED -
           30     from VOLUMES -
           40     where -
           50     DEVCLASS_NAME='TAPECLASS' -
           55     and -
           60     PCT_UTILIZED<$1 -
           70     and -
           75     STATUS like 'FILLING' -
           80     order by 3 asc

Open in new window

Run this script e.g. with

RUN scriptname 3

This will show you all "FILLING" tapes with a uitilization lower than 3%.
(Again I used "TAPECLASS" for the tape device class name - please adjust.)

If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Have fun with TSM!
0
 
LVL 61

Expert Comment

by:gheist
ID: 37745340
If you collocate by group you really need a disk pool where node data goes first, after staging/migration will orderly write data on 1 tape per collocation group.

Q STG F=D # will report media in a pool.
If there are many such tapes you could run "MOVE DATA" to empty them by moving their content to other volumes in the same pool.

Open in new window

0
 

Author Comment

by:virgo0880
ID: 37748220
I ran that query and found that many tapes in one of my copy tape pool is underutilized. So, can I perform the move data on copy tape pool just like we do it on primary tape pool or whether there is a seperate procedure for the same. I am attaching the file for the reference containing output of query. Let me know what can be done in this case.

virgo
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37748276
You can move files from a copy storage pool volume to volumes in the same copy storage pool, that's no problem.

But why should the volumes just in this copy pool be poorly utilized?
Does this storage pool differ in regard to the COLLOCATION setting?

Btw, seems you forgot to attach the output file.
0
 

Author Comment

by:virgo0880
ID: 37748293
this time with the file. Also, collocation details for the storage pool is as follows :


STGPOOL_NAME           COLLOCATE              NUMSCRATCHUSED
------------------     ------------------     --------------
CBACK6                 NO                                254
TBACK6                 NO                                241
CBACK5                 NO                                137
TBACK5                 NO                                120
CBACK2                 NO                                113
CBACK1                 NO                                101
TBACK2                 NO                                101
CBACK3                 NO                                 19
TBACK1                 NO                                 15
TBACK33                NODE                               15
TBACK11                NO                                  5


I dont see collocation is on for cback1, but still it is showing lots of underutilized tapes.
util-query.txt
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37748349
Strange.

Is CBACK1 the backup target for many primary pools, and do you run the BACKUP STGPOOL jobs in parallel, with a huge MAXPROCESS setting?
0
 
LVL 61

Assisted Solution

by:gheist
gheist earned 100 total points
ID: 37748590
The damage is done in the beginning when TSM allocates tape per node. you need to explicitly move data out from underutilised tapes once you get collocation groups right.
0
 

Author Comment

by:virgo0880
ID: 37748693
How can I check for which primary pool CBACK1 is the target pool?

And also when I see my daily DRM script, the backup stg jobs are running with maxprocess of 4.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37748800
>> How can I check for which primary pool CBACK1 is the target pool? <<

That's specified in your backup stgp jobs, nowhere else.

4 processes doesnt's sound too exaggerated, unless you use to run many jobs in parallel.
If you have indeed many such jobs consider specifying WAIT=YES.

Collocation doesn't seem to be the culprit, because there's just one pool with collocation enabled, and this one has just 15 volumes.

I'd suggest as a first step reducing the number of volumes in CBACK1 by means of MOVE DATA to then watch carefully whether this number would again grow enormously,
0
 

Author Comment

by:virgo0880
ID: 37749076
Is it ok to directly run "move data vol" on the cback1 underutilized volumes. Whether it will create any problem for the volumes which are showing as offsite or unavailable?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37749162
Yes, cou can move files from an offsite volume in a copy storage pool.
Because the offsite volume cannot be mounted, the TSM server obtains the files
that are on the offsite volume from the primary storage pool (or perhaps from another copy storage pool, if there is one containing the data).
These files are then written to the destination volumes in the original copy storage pool
and the "offsite" volume will become logically empty that way.

Unavailable volumes cannot be the source of MOVE DATA, I think.
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 

Author Comment

by:virgo0880
ID: 37749284
I see there are total 78 volumes in cback1 which are utilized less that 3%, but all of them Access mode is showing as offsite. So if I start move data for them the data will be moved to other tapes in the same pool and whether it will mark that tape as scratch/empty.

If it is marking it as empty, then whether I can recall them immediately from Iron mountain and use them as a scratch tapes?

Also, move data will move the data to the other tapes in the cback1 pool, so in that case how my data will be maintained at offsite?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37749334
Yes, the tapes will be empty once MOVE DATA has completed.

A MOVE DRMEDIA TOSTATE=ONSITER operation will bring them back onsite immediately.

Data which have been subject to MOVE DATA will no longer be offsite.
The next MOVE DRMEDIA TOSTATE=VAULT operation will bring them (their volumes, that is) offsite again.
0
 

Author Comment

by:virgo0880
ID: 37749720
we have a script running at 4.15 pm daily which requests for empty tapes from onsite. In this script the command is :

q drmedia wherestate=vaultr

whether using this command it will recall that empty tape. I see that tape when I execute the command in TSM. So my question here is, if I create a script to move data and schedule it so that it will move data from under utilized tapes and then mark them as empty so when tomorrow my tape request script runs,  it should recall all that tapes ... right?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37750054
"Q" DRMEDIA will obviously not move any tapes, but yes, it will display the now empty tapes as "vault retrieveable".

A subsequent "MOVE DRMEDIA TOSTATE=ONSITER"  will bring the tapes back then.
0
 

Author Comment

by:virgo0880
ID: 37750080
ok, got it and thanks for all that help and guidance for this issue.

virgo
0
 

Author Comment

by:virgo0880
ID: 37778639
wmp,

Ok, so i was going through my DRM script and found that there is following entry which deletes the volume history older that 32 days.

DEL VOLHIST TODATE=TODAY-32 type=all

 If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. can you confirm the same. Also what is "type=all" here?

virgo
0
 
LVL 61

Expert Comment

by:gheist
ID: 37779118
volume type - scratch/online/offsite  etc
0
 

Author Comment

by:virgo0880
ID: 37779136
thanks gheist...? but I need answer on my other part of the question also.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 37779412
>> If I disable the delete volhist option, I can use it somewhat to find out how many tapes were used using the query which you gave in comment id 37741142. <<

Yes, but you'll have to wait until a history spanning the desired time range has accumulated, deleted volhistory records will of course never come back.

TYPE=ALL means deleting history information about:

- Backupsets
- DB Backups
- DB Snapshots
- DB Rpf (= DB backups + Recovery plans)
- Export Volumes
- Library Client (remote) Volumes
- RP Files, RP Snapshots (Recovery plans)
- STGNEW (volumes added to stgpools e.g. from the scratch pool)
- STGDELETE (volumes returned to the scratch pool or deleted)
- STGREUSE (empty volumes being reused, valid for stgpools which are not allowed to use scratch volumes but have fixed volumes instead).
0
 

Author Comment

by:virgo0880
ID: 37788495
Thanks for the explanation. I will verify my configuration and accordingly configure the volume history.

Thanks
virgo
0
 
LVL 61

Assisted Solution

by:gheist
gheist earned 100 total points
ID: 37788856
You also need to eliminate underused volumes after you set more or less logical collocation groups. You can use "MOVE NODED" to move particular nodes in same collocation group away from current tapes, so that collocation applies, and do so until target volume reaches 50%....
0
 

Author Comment

by:virgo0880
ID: 37789089
I am not using collocation on my storage pools, infact I disabled it one on of my storage pools due to which there were so many underutilized tapes, on which I performed move data and now I am using them for scratch. I freed up nearly 80-90 tapes.

For future tape growth forecasting, I am looking at Servergraph to see if there are any options or reports that can be configured.

virgo
0
 
LVL 61

Expert Comment

by:gheist
ID: 37791699
If you switch off collocation you lose ability to restore fast. Having same node on set of tapes less than you have drives makes restore optimal. I think MMC reporting component has some reporting configuration (external to TSM) which emits global report and per-node reports to node managers (it takes time to get it right, look at locally configured mail relay and ask your mail admin for more info, it will merely send a mail per day per node)
0
 
LVL 61

Expert Comment

by:gheist
ID: 37791703
PS that is why i recommend the per-node approach, at least starting with services facing customers and services facing whole enterprise.
once there is downtime you do not have much breathing space to improve architecture under hood.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Replication has always been one of those technologies that people run scared from. The main reason is usually cost. When you think of replication, your mind drifts to solutions that replicate from one disk frame to another using block level technolo…
Storage devices are generally used to save the data or sometime transfer the data from one computer system to another system. However, sometimes user accidentally erased their important data from the Storage devices. Users have to know how data reco…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now