Conference decwet::networker

This is the error we are getting on Monday nights. 

Jan 14 03:07:57 qitudftp syslog: NetWorker savegroup: (notice)
PrcOraGroupDaily 
completed, 2 clients (qitduprc Failed)
Jan 14 03:07:57 qitudftp syslog: Start time:   Tue Jan 14 01:01:49 1997
Jan 14 03:07:57 qitudftp syslog: End time:     Tue Jan 14 03:07:57 1997
Jan 14 03:07:57 qitudftp syslog: 
Jan 14 03:07:57 qitudftp syslog: --- Unsuccessful Save Sets ---
Jan 14 03:07:57 qitudftp syslog: 
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline SQL*DBA: Release 7.1.4.1.1 - Production on Tue
Jan 
14 01:01:08 1997
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline Copyright (c) Oracle Corporation 1979, 1994.  All
r
ights reserved.
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline Oracle7 Server Release 7.1.4.1.1 - Production
Relea
se
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline With the distributed option
Jan 14 03:07:57 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline PL/SQL Release 2.1.4.0.0 - Production
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline SQLDBA> Connected.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline SQLDBA> Database closed.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline Database dismounted.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline ORACLE instance shut down.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline SQLDBA> 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline SQL*DBA complete.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline Oracle7 Enterprise Backup Utility: Release
2.0.12.4
.0 - Production on Tue Jan 14 01:01:19 1997
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline Copyright (c) Oracle Corporation 1979, 1994.  All
r
ights reserved.
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline 
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline CORE Version 3.5.2.0.0 - Production
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline NLSRTL Version 3.2.2.0.0 - Production
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline  
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline BACKUP job SUCCESSFUL
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_testnat_offline  
Jan 14 03:07:58 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_ber21_offline SQL*DBA complete.
Jan 14 03:08:13 qitudftp syslog: NetWorker index: (notice) nsrim has finished
ch
ecking the media db
Jan 14 03:17:01 qitudftp syslog: NetWorker media: (info) tz87 tape PRCFMo.01
wil
l be over-written
Jan 14 03:17:05 qitudftp syslog: NetWorker savegroup: (info) starting PRC
(with 
2 clients)
Jan 14 03:17:10 qitudftp syslog: NetWorker media: (info) suggest mounting
PRCFFr
.02 for backup to pool 'FileSystemPRC'
Jan 14 03:17:11 qitudftp syslog: NetWorker media: (waiting) backup to pool
'File
SystemPRC' waiting for 2 writable backup tapes
Jan 14 03:32:14 qitudftp syslog: NetWorker media: (critical) backup to pool
'Fil
eSystemPRC' waiting for 1 writable backup tape
Jan 14 03:54:10 qitudftp syslog: NetWorker media: (alert) backup to pool
'FileSy
stemPRC' waiting for 1 writable backup tape
Jan 14 05:10:42 qitudftp syslog: NetWorker savegroup: (notice) PRC completed,
2 
clients (All Succeeded)


Running the same backup scripts for Tuesday the result is.
Jan 15 00:49:17 qitudftp syslog: NetWorker savegroup: (notice)
PrcOraGroupDaily 
completed, 2 clients (All Succeeded)
Jan 15 00:49:17 qitudftp syslog: Start time:   Tue Jan 14 23:10:24 1997
Jan 15 00:49:17 qitudftp syslog: End time:     Wed Jan 15 00:49:17 1997
Jan 15 00:49:17 qitudftp syslog: 
Jan 15 00:49:17 qitudftp syslog: --- Successful Save Sets ---
Jan 15 00:49:17 qitudftp syslog: 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline SQL*DBA: Release 7.1.4.1.1 - Production on Wed Jan
15 
00:40:58 1997
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline Copyright (c) Oracle Corporation 1979, 1994.  All
righ
ts reserved.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline Oracle7 Server Release 7.1.4.1.1 - Production
Release
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline With the distributed option
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline PL/SQL Release 2.1.4.0.0 - Production
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline SQLDBA> Connected.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline SQLDBA> Database closed.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline Database dismounted.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline ORACLE instance shut down.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline SQLDBA> 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline SQL*DBA complete.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline Oracle7 Enterprise Backup Utility: Release
2.0.12.4.0 
- Production on Wed Jan 15 00:41:06 1997
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline Copyright (c) Oracle Corporation 1979, 1994.  All
righ
ts reserved.
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline 
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline CORE Version 3.5.2.0.0 - Production
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline NLSRTL Version 3.2.2.0.0 - Production
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline  
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline BACKUP job SUCCESSFUL
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_comp_offline  
Jan 15 00:49:17 qitudftp syslog: *
qitduprc:/usr/oracle/obackup/admin/backup/gro
up/daily/bck_bnat1_offline 
Jan 15 00:49:17 qitudftp syslog: 
Jan 15 00:49:30 qitudftp syslog: NetWorker index: (notice) nsrim has finished
ch
ecking the media dbJan 15 00:49:31 qitudftp syslog: NetWorker index: (notice)
nsrck is cross-checki
ng index for qitudftp
Jan 15 00:50:35 qitudftp syslog: NetWorker index: (notice) nsrck has completed
c
ross-check
Jan 15 01:09:43 qitudftp syslog: NetWorker savegroup: (info) starting PRC
(with 
2 clients)
Jan 15 01:09:46 qitudftp syslog: NetWorker media: (waiting) backup to pool
'File
SystemPRC' waiting for 1 writable backup tape
Jan 15 01:09:46 qitudftp syslog: NetWorker media: (waiting) backup to pool
'File
SystemPRC' waiting for 2 writable backup tapes
Jan 15 01:12:58 qitudftp syslog: NetWorker media: (info) tz87 tape PRCFTu.01
wil
l be over-written
Jan 15 01:13:07 qitudftp syslog: NetWorker media: (info) suggest mounting
PRCFFr
.02 for backup to pool 'FileSystemPRC'
Jan 15 01:24:51 qitudftp syslog: NetWorker media: (critical) backup to pool
'Fil
eSystemPRC' waiting for 1 writable backup tape
Jan 15 01:46:54 qitudftp syslog: NetWorker media: (alert) backup to pool
'FileSy
stemPRC' waiting for 1 writable backup tape
Jan 15 03:05:46 qitudftp syslog: NetWorker savegroup: (notice) PRC completed,
2 
clients (All Succeeded)


This is the trend every week.

Can somebody help. According to the oracle backup logs the backup do go
through.

Thanks Robert.




[Posted by WWW Notes gateway]

I renamed the topic.

When you're asked for a TITLE, it's a title for
the question you're asking, not what DEC
think's you do (i.e., "Unix Support Engineer").

Mark

Telkom has upgraded EBU to 2.0.12.4.0 and is trying to do scheduled
 saves using DMOV1.0 or V1.0A -- this will not work. Read 71.6 for details.

Are you sure the Database has been backed up???  Have you seen savesets with
 ID's like "123456789876" being saved in the NetWorker GUI (nwadmin)? or 
 are you only seeing the obackup script and the index file being saved?
 

Hi Mark,

Thanks for fixing my title, it's the first time I'm using notes.

Kevin,

According to note 71.6 EBU 2.0.12.4 do work with DMOv1.0 if the backup are
being run manually.
That is the way they do it at Telkom.

I hope this will help.

Thanks Robert.

[Posted by WWW Notes gateway]

Robert - 

 I'm confused because .0 shows a savegroup completion notification. This 
 ONLY comes from running a savegroup on the NetWorker Server, which is
 like trying to fire up a scheduled save (it will not matter to Oracle how
 the save started, just that it was initiated from the NetWorker Server).

  When #71.6 mentioned manual saves (I may need to clarify this!) it meant
 that the saves MUST be initiated FROM the NetWorker Client (either via
 hand, or via cron)... for example:

	user-oracle> obackup foobar.obk

 is the only way EBU 2.0.12.4 will let you get into the database for saves
 (because you happened to be "oracle" when the obackup process started).
 To speed up initialization, set the environment variables NSR_SERVER, and
 if desired NSR_GROUP (and insure that your NetWorker Server has that group 
 defined, and perhaps a pool with that group as it's only item. See Admin Guide
 fro details, please).

  The clues I picked up on were buried in the text stream (I'll extract):

SQL*DBA: Release 7.1.4.1.1 - Production on Tue Feb 11 01:34:30 1997
Copyright (c) Oracle Corporation 1979, 1994.  All rights reserved.
Oracle7 Server Release 7.1.4.1.1 - Production Release
With the distributed option
PL/SQL Release 2.1.4.0.0 - Production
SQLDBA> Password:
ORA-01031: insufficient privileges          <<-----first clue
SQLDBA>
 DBA-00302: not connected to a database     <<----probably tns failing
SQL*DBA complete.
Oracle7 Enterprise Backup Utility: Release 2.0.12.4.0 - Production
                                           ^^^^^^^^^^^<<<----final clue
  on Tue Feb 1101:34:30 1997
BACKUP job FAILED
obkasm: "/usr/bin/obackup" (PID 32629) exited with code 8
obkasm: the database backup may have failed.  Please check the log and trace

  The rest of this is obkasm basically letting you know it tried to fire up 
 obackup, and could not.

Kevin,

To clarify, in .0 we found the problem after I've posted the note. The
customer
changed the permissions and user. The customers main problem is on Monday
nights
the errors we are receiving, even if the backup worked.

I will check the way they are doing their backups and get back to you.

What I did was to add more notifications to get a better log for Monday
nights.
What we have to do is to wait for Monday. I will start a trace as well on
Monday
when they start the backup.

The reason we are getting a "Insufficient priv.... " error is:

They do not use user oracle in their scripts for security reasons.
The backup user is fsbackup and in oracle the dba gave it the privilege to
start
and shut the databases.
The "Insufficient priv.." is because only user sys in oracle can do a SHOW
SGA.
The customer is not worried about that so much, they have asked oracle if it
can 
be taken out.

The precmd:

#!/bin/sh
ORACLE_HOME=/usr/oracle
export ORACLE_HOME
ORACLE_SID=comp
export ORACLE_SID
/usr/oracle/bin/sqldba lmode=y << EOF
connect fsbackup/fsbackup as sysoper
shutdown immediate
EOF

The postcmd:#!/bin/sh
ORACLE_HOME=/usr/oracle
export ORACLE_HOME
ORACLE_SID=comp
export ORACLE_SID
/usr/oracle/bin/sqldba lmode=y << EOF
connect fsbackup/fsbackup as sysoper
startup
EOF

The nsr.cf file on the client:

export OBK_HOME=/usr/oracle/obackup
export NSR_SERVER=qitudftp
export LD_LIBRARY_PATH=$OBK_HOME/lib
export PATH=$PATH:$OBK_HOME/bin
export SHLIB_PATH=/usr/shlib

Thanks so far 
Robert
[Posted by WWW Notes gateway]

Just a small nit, Robert;
You're corresponding with Bruce, not Kevin.

Bruce is the Oracle/DMO expert.

Kevin

Robert --

  I got your file OK via email. 

  Q1 -- It is the NetWorker Servers /nsr/logs/messages file, right??

  Q2 -- are you going to send me the obackup trace and log file??

  Q3 -- what is the error message I'm supposedly looking for?? Please
	do NOT post the savegroup completion - describe this symptom or
	error in 1 sentence.

thanks.  Bruce.

Bruce,

The error to look for is:
Monday nights client qitudprc report a failure on only one database. Futher
down
it reported "backup job successfull". The backups do finish. The customer
don't
want to see the error.

Robert.
[Posted by WWW Notes gateway]

Robert --

  I'm sorry, there's not much I feel I can do at this point. I've searched
 the MESSAGES file you emailed for 'ORA-' (found none), looked everywhere for
 "qitduprc" for any error and found only 1 (savegroup abandoned), looked in
 basenote for more clues, and could not find those in MESSAGES either. I tried
 searching for "qitudprc" and found no occurances.

   Given that this is DMO V1, I'd suggest that the next monday night customer
 sees this fail, they add the requested lines into the obackup script

	log="..."
	trace="..."

 and add into nsr.cf the "NSR_DEBUG_FILE /path", run the backup again, and
 ship me those files (the log, trace, and debug file).

On another topic: I noticed that Telkom is doing scheduled backups using DMOV1
 and EBU 2.0.12.4... here I've been telling people they can't do that, and
 Telkom is off *doing* it... care to share??

Also, would they care to Field Test DMOv2?

Robert -- 

  well, I researched the log files (all of them), and re-read this note thread,
 and have nothing to add. I didn't ask for the NetWorker log file so I could
 see the "error message" that is bothering Telkom, and the obackup log
 shows everything worked OK, as did the DMO logs you supplied. Perhaps, if I
 got the matching NetWorker Log with the "error message" highlited so I knew
 what was bothering Telkom, I could pursue this further.

   I'd like to point out that DMO V2 is designed to work with EBU 2.0.10
 and/or EBU 2.0.12 and NetWorker 3.2 of higher. DMO V1 is only documented as
 working with EBU 2.0.10 and NetWorker V3.2.

   I strongly recommend Telkom evaluate (a) become a Field Test Site for DMO
 V2, or (b) live with this error until DMO V2 ships and upgrade.

 Thank you for sending email reminding me this was due back to you and Telkom.
 Last week was understaffed, and over busy.