Solved

Slow RMAN backups Oracle 10g with Veritas Netbackup 6.5.3

Posted on 2009-04-06
6
3,674 Views
Last Modified: 2013-12-18
Hi, I am a Window Systems Admin and I am in charge of the Oracle 10g database backups.  Here is the setup:
Backup Server:
Windows 2003, 2.0GHz x2,  2GB RAM, 1Gbps
Veritas netbackup 6.5.3
DLT2 drive tops out at about ~5-6MB/sec (Compression is off)

Oracle Server:
Windows 2003, 2.0GHz x4, 4GB RAM, 1Gbps
Oracle 10g using RMAN to backups, auto control file, 40GB database

We pretty much used the RMAN script from veritas located in the DBext folder on the client.  Here are the stats on the backups I am running.
Windows file system backup to tape:   ~5MB/s
Oracle RMAN to tape: ~2MB/s
Oracle RMAN to HDD (Veritas virtual tape drive) ~10MB/s

My quersion is, why are the oracle RMAN tape backups so slow?  My second question would be, why are my Backups to tape so slow, when I can copy a file over the network @ ~30MB/s over gigabit.
@REM $Header: hot_database_backup.cmd,v 1.2 2003/09/11 21:17:14 $
 
@REM bcpyrght
@REM *************************************************************************** 
@REM * $VRTScprght: Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved $ * 
@REM *************************************************************************** 
@REM ecpyrght
@REM
@REM ---------------------------------------------------------------------------
@REM  			hot_database_backup.cmd modified by Ram Akella 05/24/06
@REM ---------------------------------------------------------------------------
@REM This script uses Recovery Manager to take a hot (inconsistent) database
@REM backup. A hot backup is inconsistent because portions of the database are
@REM being modified and written to the disk while the backup is progressing.
@REM You must run your database in ARCHIVELOG mode to make hot backups. 
@REM
@REM NOTE information for running proxy backups has been included.  These 
@REM information sections begin with a comment line of PROXY 
@REM ---------------------------------------------------------------------------
 
@setlocal ENABLEEXTENSIONS
 
@REM ---------------------------------------------------------------------------
@REM No need to echo the commands. modified by Ram Akella 03/24/05
@REM Added echo delete input all to remove all archive logs after backups by RAM 0n 06/22/06
@REM This script works with cluster name placed on both servers orap01 and orap02 
@REM when cluster failed over 07/20/06
@REM ---------------------------------------------------------------------------
 
@echo off
 
@REM ---------------------------------------------------------------------------
@REM Put output in the same filename, different extension.
@REM ---------------------------------------------------------------------------
@echo off
set curndate=%date%
set day=%curndate:~4,2%
set date=%curndate:~7,2%
set year=%curndate:~10,4% 
@set RMAN_LOG_FILE="C:\Program Files\VERITAS\NetBackup\Temp\oracle_daily_prod_cl_archivelogs_del7days-%day%%date%%year%.log"
 
@REM ---------------------------------------------------------------------------
@REM You may want to delete the output file so that backup information does
@REM not accumulate.  If not, delete the following command.
@REM ---------------------------------------------------------------------------
 
@REM ---------------------------------------------------------------------------
@REM Replace H:\oracle\ora81, below, with the Oracle home path.
@REM ---------------------------------------------------------------------------
 
@set ORACLE_HOME=C:\Oracle\Ora10g
 
@REM ---------------------------------------------------------------------------
@REM Replace ora81, below, with the Oracle SID.
@REM ---------------------------------------------------------------------------
 
@set ORACLE_SID=dbnamehere
 
@REM ---------------------------------------------------------------------------
@REM Replace sys/manager, below, with the target connect string.
@REM ---------------------------------------------------------------------------
 
@set TARGET_CONNECT_STR=user/password 
@REM ---------------------------------------------------------------------------
@REM Set the Oracle Recovery Manager.
@REM ---------------------------------------------------------------------------
 
@set RMAN=%ORACLE_HOME%\bin\rman.exe
 
@REM ---------------------------------------------------------------------------
@REM PROXY
@REM For a PROXY backup, uncomment the line below and replace the value. 
@REM
@REM       NB_ORA_PC_STREAMS - specifies the number of parallel backup streams
@REM                           to be started.
@REM ---------------------------------------------------------------------------
@REM @set NB_ORA_PC_STREAMS=3
 
 
@REM ---------------------------------------------------------------------------
@REM Log the start of this scripts.
@REM ---------------------------------------------------------------------------
 
@for /F "tokens=1*" %%p in ('date /T') do @set DATE=%%p %%q
@for /F %%p in ('time /T') do @set DATE=%DATE% %%p
 
@echo ==== started on %DATE% ==== >> %RMAN_LOG_FILE%
@echo Script name: %0 >> %RMAN_LOG_FILE%
 
@REM ---------------------------------------------------------------------------
@REM Several RMAN commands use time parameters that require NLS_LANG and 
@REM NLS_DATE_FORMAT to be set. This example uses the standard date format.
@REM Replace below with the desired language values.
@REM ---------------------------------------------------------------------------
 
@set NLS_LANG=american
@set NLS_DATE_FORMAT=YYYY-MM-DD:hh24:mi:ss
 
@REM ---------------------------------------------------------------------------
@REM Print out environment variables set in this script.
@REM ---------------------------------------------------------------------------
 
@echo #                                       >> %RMAN_LOG_FILE%
@echo   RMAN  :  %RMAN%                       >> %RMAN_LOG_FILE%
@echo   NLS_LANG  :  %NLS_LANG%               >> %RMAN_LOG_FILE%
@echo   ORACLE_HOME  :  %ORACLE_HOME%         >> %RMAN_LOG_FILE%
@echo   ORACLE_SID  :  %ORACLE_SID%           >> %RMAN_LOG_FILE%
@echo   NLS_DATE_FORMAT  :  %NLS_DATE_FORMAT% >> %RMAN_LOG_FILE%
@echo   RMAN_LOG_FILE  :  %RMAN_LOG_FILE%     >> %RMAN_LOG_FILE%
 
@REM ---------------------------------------------------------------------------
@REM PROXY
@REM For a PROXY backup, uncomment the line below. 
@REM ---------------------------------------------------------------------------
@REM @echo   NB_ORA_PC_STREAMS  :  %NB_ORA_PC_STREAMS%     >> %RMAN_LOG_FILE%
 
@REM ---------------------------------------------------------------------------
@REM Print out environment variables set in bphdb.
@REM ---------------------------------------------------------------------------
 
@echo   NB_ORA_SERV  :  %NB_ORA_SERV%                     >> %RMAN_LOG_FILE%
@echo   NB_ORA_FULL  :  %NB_ORA_FULL%                     >> %RMAN_LOG_FILE%
@echo   NB_ORA_INCR  :  %NB_ORA_INCR%                     >> %RMAN_LOG_FILE%
@echo   NB_ORA_CINC  :  %NB_ORA_CINC%                     >> %RMAN_LOG_FILE%
@echo   NB_ORA_CLASS  :  %NB_ORA_CLASS%                   >> %RMAN_LOG_FILE%
 
@REM ---------------------------------------------------------------------------
@REM We assume that the database is properly opened. If desired, this would
@REM be the place to verify that.
@REM ---------------------------------------------------------------------------
 
@REM ---------------------------------------------------------------------------
@REM If this script is executed from a NetBackup schedule, NetBackup
@REM sets an NB_ORA environment variable based on the schedule type.
@REM For example, when:
@REM     schedule type is                BACKUP_TYPE is
@REM     ----------------                --------------
@REM Automatic Full                      INCREMENTAL LEVEL=0
@REM Automatic Differential Incremental  INCREMENTAL LEVEL=1
@REM Automatic Cumulative Incremental    INCREMENTAL LEVEL=1 CUMULATIVE
@REM
@REM For user initiated backups, BACKUP_TYPE defaults to incremental
@REM level 0 (Full).  To change the default for a user initiated
@REM backup to incremental or incrementatl cumulative, uncomment
@REM one of the following two lines.
@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1"
@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1
@REM Ram modified the script to run the incremental backups on 06/01/05 
@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1 CUMULATIVE"
@REM
@REM Note that we use incremental level 0 to specify full backups.
@REM That is because, although they are identical in content, only
@REM the incremental level 0 backup can have incremental backups of
@REM level > 0 applied to it.
@REM ---------------------------------------------------------------------------
 
@REM ---------------------------------------------------------------------------
@REM What kind of backup will we perform.
@REM ---------------------------------------------------------------------------
 
@if "%NB_ORA_FULL%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=0
@if "%NB_ORA_INCR%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=1
@if "%NB_ORA_CINC%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=1 CUMULATIVE
@if NOT DEFINED BACKUP_TYPE @set BACKUP_TYPE=INCREMENTAL Level=0
 
@REM ---------------------------------------------------------------------------
@REM Call Recovery Manager to initiate the backup. This example does not use a
@REM Recovery Catalog. If you choose to use one, remove the option, nocatalog,
@REM from the rman command line below and add a 
@REM 'rcvcat <userid>/<passwd>@<tns alias>' statement.
@REM
@REM  NOTE WHEN USING TNS ALIAS: When connecting to a database
@REM  using a TNS alias, you must use a send command or a parms operand to 
@REM  specify environment variables.  In other words, when accessing a database
@REM  through a listener, the environment variables set at the system level are not 
@REM  visible when RMAN is running.  For more information on the environment
@REM  variables, please refer to the NetBackup for Oracle Admin. Guide.
@REM
@REM ---------------------------------------------------------------------------
 
@REM ---------------------------------------------------------------------------
@REM PROXY
@REM For a PROXY backup, you must use a send command to specify
@REM the NB_ORA_PC_STREAMS environment variable. For example,
@REM echo ALLOCATE CHANNEL ch01 TYPE 'SBT_TAPE';
@REM echo SEND 'NB_ORA_PC_STREAMS=%%NB_ORA_PC_STREAMS%%'; 
@REM 
@REM %BACKUP_TYPE% must also be removed and replaced with the PROXY parameter 
@REM in the RMAN section associated with the data files.  For example,
@REM echo BACKUP
@REM echo       PROXY
@REM echo       FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'
@REM echo       DATABASE;
@REM            .
@REM            . 
@REM  Note that the controlfiles and archivelogs are not backed up using proxy
@REM  copy method. Rman will initiate non-proxy copy sessions to backup the
@REM  controlfile and archivelogs.
@REM ---------------------------------------------------------------------------
@(
echo RUN {
echo ALLOCATE CHANNEL ch00 
echo     TYPE 'SBT_TAPE';
echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';
echo BACKUP
echo       %BACKUP_TYPE%
echo       FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'
echo       DATABASE;
echo RELEASE CHANNEL ch00;
echo ALLOCATE CHANNEL ch00 TYPE 'SBT_TAPE';
echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';
echo # Backup all archive logs
echo sql 'alter system archive log current';
echo BACKUP
echo       FILESPERSET 40
echo       FORMAT 'arch-s%%s-p%%p'
echo       ARCHIVELOG ALL
echo ARCHIVELOG UNTIL TIME 'SYSDATE-7' DELETE ALL INPUT;
echo RELEASE CHANNEL ch00;
echo ALLOCATE CHANNEL ch00 TYPE 'SBT_TAPE';
echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';
echo # The above one is working 
echo BACKUP
echo     FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'
echo     CURRENT CONTROLFILE;
echo RELEASE CHANNEL ch00;
echo }
) | %RMAN% target %TARGET_CONNECT_STR% nocatalog msglog '%RMAN_LOG_FILE%' append
 
@set ERRLEVEL=%ERRORLEVEL%
 
@REM ---------------------------------------------------------------------------
@REM NetBackup (bphdb) stores the name of a file in an environment variable, 
@REM called STATUS_FILE. This file is used by an automatic schedule to
@REM communicate status information with NetBackup's job monitor. It is up to 
@REM the script to write a 0 (passed) or 1 (failure) to the status file.
@REM modified by Ram Akella 05/24/06.
@REM ---------------------------------------------------------------------------
 
@if %ERRLEVEL% NEQ 0 @goto err
 
@set LOGMSG=ended successfully
@if "%STATUS_FILE%" EQU "" goto end
@echo 0 > "%STATUS_FILE%"
@goto end
 
:err
@set LOGMSG=ended in error
@if "%STATUS_FILE%" EQU "" @goto end
@echo 1 > "%STATUS_FILE%"
 
:end
 
@REM ---------------------------------------------------------------------------
@REM Log the completion of this script. modified by Ram Akella 03/24/05
@REM ---------------------------------------------------------------------------
 
@for /F "tokens=1*" %%p in ('date /T') do @set DATE=%%p %%q
@for /F %%p in ('time /T') do @set DATE=%DATE% %%p
 
@echo #  >> %RMAN_LOG_FILE% 
@echo %==== %LOGMSG% on %DATE% ==== >> %RMAN_LOG_FILE%
@endlocal
@REM End of Main Program -----------------------------------------------------

Open in new window

0
Comment
Question by:Unisys1
  • 4
  • 2
6 Comments
 
LVL 23

Expert Comment

by:bhanukir7
ID: 24085737
hi,

if your file system backup itself is giving you 5 MB no wonder that your application specific data backup would be half the amount.

The reason is that the application processes the data and sends it to the backup up application which in turn sends it to the tape drive.

one bottle neck i would say is your DLT2 tape drive.  

Try to run a file system backup and the oracle RMAN backup to a D2D folder and verify if you are getting a better throughput. You should be getting a better throughput when you run a backup to the disk as the data is written randomly which is faster when compared to a SCSI device which is done in a sequential manner.

if you intend to finish the backups faster then you can possible run the backups to disk and migrate the same to tape.

this would certainly bring down your backup window.

bhanu
0
 

Author Comment

by:Unisys1
ID: 24087314
I have tried D2D backup & I can achieve ~10MB/s, which is still super slow.  When I restored the system at DR, I was restoring to a fresh build of the DB @ 15MB/s using a SDLT (much faster) drive & an original DLT2 tape.  I'm thinking that it is an Oracle settings, But not sure what that setting might be.  Both backup & Restore are single channel.
0
 
LVL 23

Expert Comment

by:bhanukir7
ID: 24088532
hi,

you can increase the channels(streams) of RMAN provided you are running the backup when the DATABASE is not in much use.

How much time does it take for you when you run a RMAN script backup to local disk. Does it finish of faster. When i say RMAN backup i am referring to disk based backup using RMAN and not any backup solution.

i am not sure how big is your oracle DB but 15 MB also sounds very very low to me.

try running perfmon on the disksubsystem when you are running backup to verify if you have any memory/disk/processor bottlenecks.

bhanu
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 

Author Comment

by:Unisys1
ID: 24088813
When I increase the channels to 2, it requires 2 tapes.  Is this normal?
0
 

Author Comment

by:Unisys1
ID: 24130387
I have decided to run backups as compressed, this is bringing speeds up to target.  I will report back once we convert to LTO4, this week.
0
 

Accepted Solution

by:
Unisys1 earned 0 total points
ID: 24188709
SInce I have not received any response, I've decided to just stick with Compressed backups.
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Free HD cloner 11 82
update statement in oracle 9 29
Fill Date time Field 12 23
VMware vSAN 6.5 usage example and caveats 11 76
How to update Firmware and Bios in Dell Equalogic PS6000 Arrays and Hard Disks firmware update.
Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
This tutorial will walk an individual through the process of configuring basic necessities in order to use the 2010 version of Data Protection Manager. These include storage, agents, and protection jobs. Launch Data Protection Manager from the deskt…
This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question