Solved

Slow RMAN backups Oracle 10g with Veritas Netbackup 6.5.3

Posted on 2009-04-06
6
3,628 Views
Last Modified: 2013-12-18
Hi, I am a Window Systems Admin and I am in charge of the Oracle 10g database backups.  Here is the setup:
Backup Server:
Windows 2003, 2.0GHz x2,  2GB RAM, 1Gbps
Veritas netbackup 6.5.3
DLT2 drive tops out at about ~5-6MB/sec (Compression is off)

Oracle Server:
Windows 2003, 2.0GHz x4, 4GB RAM, 1Gbps
Oracle 10g using RMAN to backups, auto control file, 40GB database

We pretty much used the RMAN script from veritas located in the DBext folder on the client.  Here are the stats on the backups I am running.
Windows file system backup to tape:   ~5MB/s
Oracle RMAN to tape: ~2MB/s
Oracle RMAN to HDD (Veritas virtual tape drive) ~10MB/s

My quersion is, why are the oracle RMAN tape backups so slow?  My second question would be, why are my Backups to tape so slow, when I can copy a file over the network @ ~30MB/s over gigabit.
@REM $Header: hot_database_backup.cmd,v 1.2 2003/09/11 21:17:14 $

 

@REM bcpyrght

@REM *************************************************************************** 

@REM * $VRTScprght: Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved $ * 

@REM *************************************************************************** 

@REM ecpyrght

@REM

@REM ---------------------------------------------------------------------------

@REM  			hot_database_backup.cmd modified by Ram Akella 05/24/06

@REM ---------------------------------------------------------------------------

@REM This script uses Recovery Manager to take a hot (inconsistent) database

@REM backup. A hot backup is inconsistent because portions of the database are

@REM being modified and written to the disk while the backup is progressing.

@REM You must run your database in ARCHIVELOG mode to make hot backups. 

@REM

@REM NOTE information for running proxy backups has been included.  These 

@REM information sections begin with a comment line of PROXY 

@REM ---------------------------------------------------------------------------

 

@setlocal ENABLEEXTENSIONS

 

@REM ---------------------------------------------------------------------------

@REM No need to echo the commands. modified by Ram Akella 03/24/05

@REM Added echo delete input all to remove all archive logs after backups by RAM 0n 06/22/06

@REM This script works with cluster name placed on both servers orap01 and orap02 

@REM when cluster failed over 07/20/06

@REM ---------------------------------------------------------------------------

 

@echo off

 

@REM ---------------------------------------------------------------------------

@REM Put output in the same filename, different extension.

@REM ---------------------------------------------------------------------------

@echo off

set curndate=%date%

set day=%curndate:~4,2%

set date=%curndate:~7,2%

set year=%curndate:~10,4% 

@set RMAN_LOG_FILE="C:\Program Files\VERITAS\NetBackup\Temp\oracle_daily_prod_cl_archivelogs_del7days-%day%%date%%year%.log"

 

@REM ---------------------------------------------------------------------------

@REM You may want to delete the output file so that backup information does

@REM not accumulate.  If not, delete the following command.

@REM ---------------------------------------------------------------------------

 

@REM ---------------------------------------------------------------------------

@REM Replace H:\oracle\ora81, below, with the Oracle home path.

@REM ---------------------------------------------------------------------------

 

@set ORACLE_HOME=C:\Oracle\Ora10g

 

@REM ---------------------------------------------------------------------------

@REM Replace ora81, below, with the Oracle SID.

@REM ---------------------------------------------------------------------------

 

@set ORACLE_SID=dbnamehere

 

@REM ---------------------------------------------------------------------------

@REM Replace sys/manager, below, with the target connect string.

@REM ---------------------------------------------------------------------------

 

@set TARGET_CONNECT_STR=user/password 

@REM ---------------------------------------------------------------------------

@REM Set the Oracle Recovery Manager.

@REM ---------------------------------------------------------------------------

 

@set RMAN=%ORACLE_HOME%\bin\rman.exe

 

@REM ---------------------------------------------------------------------------

@REM PROXY

@REM For a PROXY backup, uncomment the line below and replace the value. 

@REM

@REM       NB_ORA_PC_STREAMS - specifies the number of parallel backup streams

@REM                           to be started.

@REM ---------------------------------------------------------------------------

@REM @set NB_ORA_PC_STREAMS=3

 

 

@REM ---------------------------------------------------------------------------

@REM Log the start of this scripts.

@REM ---------------------------------------------------------------------------

 

@for /F "tokens=1*" %%p in ('date /T') do @set DATE=%%p %%q

@for /F %%p in ('time /T') do @set DATE=%DATE% %%p

 

@echo ==== started on %DATE% ==== >> %RMAN_LOG_FILE%

@echo Script name: %0 >> %RMAN_LOG_FILE%

 

@REM ---------------------------------------------------------------------------

@REM Several RMAN commands use time parameters that require NLS_LANG and 

@REM NLS_DATE_FORMAT to be set. This example uses the standard date format.

@REM Replace below with the desired language values.

@REM ---------------------------------------------------------------------------

 

@set NLS_LANG=american

@set NLS_DATE_FORMAT=YYYY-MM-DD:hh24:mi:ss

 

@REM ---------------------------------------------------------------------------

@REM Print out environment variables set in this script.

@REM ---------------------------------------------------------------------------

 

@echo #                                       >> %RMAN_LOG_FILE%

@echo   RMAN  :  %RMAN%                       >> %RMAN_LOG_FILE%

@echo   NLS_LANG  :  %NLS_LANG%               >> %RMAN_LOG_FILE%

@echo   ORACLE_HOME  :  %ORACLE_HOME%         >> %RMAN_LOG_FILE%

@echo   ORACLE_SID  :  %ORACLE_SID%           >> %RMAN_LOG_FILE%

@echo   NLS_DATE_FORMAT  :  %NLS_DATE_FORMAT% >> %RMAN_LOG_FILE%

@echo   RMAN_LOG_FILE  :  %RMAN_LOG_FILE%     >> %RMAN_LOG_FILE%

 

@REM ---------------------------------------------------------------------------

@REM PROXY

@REM For a PROXY backup, uncomment the line below. 

@REM ---------------------------------------------------------------------------

@REM @echo   NB_ORA_PC_STREAMS  :  %NB_ORA_PC_STREAMS%     >> %RMAN_LOG_FILE%

 

@REM ---------------------------------------------------------------------------

@REM Print out environment variables set in bphdb.

@REM ---------------------------------------------------------------------------

 

@echo   NB_ORA_SERV  :  %NB_ORA_SERV%                     >> %RMAN_LOG_FILE%

@echo   NB_ORA_FULL  :  %NB_ORA_FULL%                     >> %RMAN_LOG_FILE%

@echo   NB_ORA_INCR  :  %NB_ORA_INCR%                     >> %RMAN_LOG_FILE%

@echo   NB_ORA_CINC  :  %NB_ORA_CINC%                     >> %RMAN_LOG_FILE%

@echo   NB_ORA_CLASS  :  %NB_ORA_CLASS%                   >> %RMAN_LOG_FILE%

 

@REM ---------------------------------------------------------------------------

@REM We assume that the database is properly opened. If desired, this would

@REM be the place to verify that.

@REM ---------------------------------------------------------------------------

 

@REM ---------------------------------------------------------------------------

@REM If this script is executed from a NetBackup schedule, NetBackup

@REM sets an NB_ORA environment variable based on the schedule type.

@REM For example, when:

@REM     schedule type is                BACKUP_TYPE is

@REM     ----------------                --------------

@REM Automatic Full                      INCREMENTAL LEVEL=0

@REM Automatic Differential Incremental  INCREMENTAL LEVEL=1

@REM Automatic Cumulative Incremental    INCREMENTAL LEVEL=1 CUMULATIVE

@REM

@REM For user initiated backups, BACKUP_TYPE defaults to incremental

@REM level 0 (Full).  To change the default for a user initiated

@REM backup to incremental or incrementatl cumulative, uncomment

@REM one of the following two lines.

@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1"

@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1

@REM Ram modified the script to run the incremental backups on 06/01/05 

@REM @set BACKUP_TYPE="INCREMENTAL LEVEL=1 CUMULATIVE"

@REM

@REM Note that we use incremental level 0 to specify full backups.

@REM That is because, although they are identical in content, only

@REM the incremental level 0 backup can have incremental backups of

@REM level > 0 applied to it.

@REM ---------------------------------------------------------------------------

 

@REM ---------------------------------------------------------------------------

@REM What kind of backup will we perform.

@REM ---------------------------------------------------------------------------

 

@if "%NB_ORA_FULL%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=0

@if "%NB_ORA_INCR%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=1

@if "%NB_ORA_CINC%" EQU "1" @set BACKUP_TYPE=INCREMENTAL Level=1 CUMULATIVE

@if NOT DEFINED BACKUP_TYPE @set BACKUP_TYPE=INCREMENTAL Level=0

 

@REM ---------------------------------------------------------------------------

@REM Call Recovery Manager to initiate the backup. This example does not use a

@REM Recovery Catalog. If you choose to use one, remove the option, nocatalog,

@REM from the rman command line below and add a 

@REM 'rcvcat <userid>/<passwd>@<tns alias>' statement.

@REM

@REM  NOTE WHEN USING TNS ALIAS: When connecting to a database

@REM  using a TNS alias, you must use a send command or a parms operand to 

@REM  specify environment variables.  In other words, when accessing a database

@REM  through a listener, the environment variables set at the system level are not 

@REM  visible when RMAN is running.  For more information on the environment

@REM  variables, please refer to the NetBackup for Oracle Admin. Guide.

@REM

@REM ---------------------------------------------------------------------------

 

@REM ---------------------------------------------------------------------------

@REM PROXY

@REM For a PROXY backup, you must use a send command to specify

@REM the NB_ORA_PC_STREAMS environment variable. For example,

@REM echo ALLOCATE CHANNEL ch01 TYPE 'SBT_TAPE';

@REM echo SEND 'NB_ORA_PC_STREAMS=%%NB_ORA_PC_STREAMS%%'; 

@REM 

@REM %BACKUP_TYPE% must also be removed and replaced with the PROXY parameter 

@REM in the RMAN section associated with the data files.  For example,

@REM echo BACKUP

@REM echo       PROXY

@REM echo       FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'

@REM echo       DATABASE;

@REM            .

@REM            . 

@REM  Note that the controlfiles and archivelogs are not backed up using proxy

@REM  copy method. Rman will initiate non-proxy copy sessions to backup the

@REM  controlfile and archivelogs.

@REM ---------------------------------------------------------------------------

@(

echo RUN {

echo ALLOCATE CHANNEL ch00 

echo     TYPE 'SBT_TAPE';

echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';

echo BACKUP

echo       %BACKUP_TYPE%

echo       FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'

echo       DATABASE;

echo RELEASE CHANNEL ch00;

echo ALLOCATE CHANNEL ch00 TYPE 'SBT_TAPE';

echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';

echo # Backup all archive logs

echo sql 'alter system archive log current';

echo BACKUP

echo       FILESPERSET 40

echo       FORMAT 'arch-s%%s-p%%p'

echo       ARCHIVELOG ALL

echo ARCHIVELOG UNTIL TIME 'SYSDATE-7' DELETE ALL INPUT;

echo RELEASE CHANNEL ch00;

echo ALLOCATE CHANNEL ch00 TYPE 'SBT_TAPE';

echo SEND 'NB_ORA_CLIENT=INPHORAP00,NB_ORA_POLICY=PROD_Oracle_Daily,NB_ORA_SERV=INPHBKPP01,NB_ORA_SCHED=Default-Application-Backup';

echo # The above one is working 

echo BACKUP

echo     FORMAT 'bk_u%%u_s%%s_p%%p_t%%t'

echo     CURRENT CONTROLFILE;

echo RELEASE CHANNEL ch00;

echo }

) | %RMAN% target %TARGET_CONNECT_STR% nocatalog msglog '%RMAN_LOG_FILE%' append

 

@set ERRLEVEL=%ERRORLEVEL%

 

@REM ---------------------------------------------------------------------------

@REM NetBackup (bphdb) stores the name of a file in an environment variable, 

@REM called STATUS_FILE. This file is used by an automatic schedule to

@REM communicate status information with NetBackup's job monitor. It is up to 

@REM the script to write a 0 (passed) or 1 (failure) to the status file.

@REM modified by Ram Akella 05/24/06.

@REM ---------------------------------------------------------------------------

 

@if %ERRLEVEL% NEQ 0 @goto err

 

@set LOGMSG=ended successfully

@if "%STATUS_FILE%" EQU "" goto end

@echo 0 > "%STATUS_FILE%"

@goto end

 

:err

@set LOGMSG=ended in error

@if "%STATUS_FILE%" EQU "" @goto end

@echo 1 > "%STATUS_FILE%"

 

:end

 

@REM ---------------------------------------------------------------------------

@REM Log the completion of this script. modified by Ram Akella 03/24/05

@REM ---------------------------------------------------------------------------

 

@for /F "tokens=1*" %%p in ('date /T') do @set DATE=%%p %%q

@for /F %%p in ('time /T') do @set DATE=%DATE% %%p

 

@echo #  >> %RMAN_LOG_FILE% 

@echo %==== %LOGMSG% on %DATE% ==== >> %RMAN_LOG_FILE%

@endlocal

@REM End of Main Program -----------------------------------------------------

Open in new window

0
Comment
Question by:Unisys1
  • 4
  • 2
6 Comments
 
LVL 23

Expert Comment

by:bhanukir7
Comment Utility
hi,

if your file system backup itself is giving you 5 MB no wonder that your application specific data backup would be half the amount.

The reason is that the application processes the data and sends it to the backup up application which in turn sends it to the tape drive.

one bottle neck i would say is your DLT2 tape drive.  

Try to run a file system backup and the oracle RMAN backup to a D2D folder and verify if you are getting a better throughput. You should be getting a better throughput when you run a backup to the disk as the data is written randomly which is faster when compared to a SCSI device which is done in a sequential manner.

if you intend to finish the backups faster then you can possible run the backups to disk and migrate the same to tape.

this would certainly bring down your backup window.

bhanu
0
 

Author Comment

by:Unisys1
Comment Utility
I have tried D2D backup & I can achieve ~10MB/s, which is still super slow.  When I restored the system at DR, I was restoring to a fresh build of the DB @ 15MB/s using a SDLT (much faster) drive & an original DLT2 tape.  I'm thinking that it is an Oracle settings, But not sure what that setting might be.  Both backup & Restore are single channel.
0
 
LVL 23

Expert Comment

by:bhanukir7
Comment Utility
hi,

you can increase the channels(streams) of RMAN provided you are running the backup when the DATABASE is not in much use.

How much time does it take for you when you run a RMAN script backup to local disk. Does it finish of faster. When i say RMAN backup i am referring to disk based backup using RMAN and not any backup solution.

i am not sure how big is your oracle DB but 15 MB also sounds very very low to me.

try running perfmon on the disksubsystem when you are running backup to verify if you have any memory/disk/processor bottlenecks.

bhanu
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 

Author Comment

by:Unisys1
Comment Utility
When I increase the channels to 2, it requires 2 tapes.  Is this normal?
0
 

Author Comment

by:Unisys1
Comment Utility
I have decided to run backups as compressed, this is bringing speeds up to target.  I will report back once we convert to LTO4, this week.
0
 

Accepted Solution

by:
Unisys1 earned 0 total points
Comment Utility
SInce I have not received any response, I've decided to just stick with Compressed backups.
0

Featured Post

Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

Join & Write a Comment

Storage devices are generally used to save the data or sometime transfer the data from one computer system to another system. However, sometimes user accidentally erased their important data from the Storage devices. Users have to know how data reco…
By default, Carbonite Server Backup manages your encryption key for you using Advanced Encryption Standard (AES) 128-bit encryption. If you choose to manage your private encryption key, your backups will be encrypted using AES 256-bit encryption.
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
This video shows how to configure and send email from and Oracle database using both UTL_SMTP and UTL_MAIL, as well as comparing UTL_SMTP to a manual SMTP conversation with a mail server.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now