[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference decwet::winnt-clusters

Title:WinNT-Clusters
Notice:Info directories moved to DECWET::SHARE1$:[NT_CLSTR]
Moderator:DECWET::CAPPELLOF
Created:Thu Oct 19 1995
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:863
Total number of notes:3478

760.0. "SQL Failover problems" by CHEFS::vfr750.reo.dec.com::harrisonb () Tue Apr 22 1997 09:18

[DEC NT Clusters V1.1]

I have a customer who has a cluster. The cluster shares & IP addrs are failing over
correctly, however he is having a few problems with SQL. He has a database that he
has on one node in the cluster, he can access it through the cluster alias, it even enrolls.
However when he fails over the group it is in, it does not appear on the other node,
and cannot be accessed from the client.

I have included the log he sent me. Any Ideas ??

PS Donna, any more on the Policies Issue I posted a few notes back ?
PPS How do I submit a PTR ?

Cheers, Brynn Harrison
T.RTitleUserPersonal
Name		DateLines
760.1Log part 1CHEFS::vfr750.reo.dec.com::harrisonbTue Apr 22 1997 09:21132
Digital Clusters for Windows NT(TM) 1.1-1222 Release (Build 1222)
Digital Equipment Corporation
Windows NT(TM) is a trademark of Microsoft Corporation.
Cluster Failover Manager Trace File
Opened on cluster tuesday node LEFT at 4/22/97 10:19:08 AM
by program D:\Program Files\Digital\Cluster\fmcore.exe

04/22/1997 10:19:08.578 tid=124 trace started
10:19:08.656 tid=124 Cfmd Server is not ready, waiting...
10:19:38.796 tid=124 Loading failover object dll fmdisk.dll...
10:19:38.890 tid=124 Disk Failover Object DLL Process Attach
10:19:38.921 tid=124 Disk Failover Object DLL Establish Linkage
10:19:38.953 tid=124 Loading failover object dll fmip.dll...
10:19:39.062 tid=124 IP Failover Object DLL Process Attach
10:19:39.109 tid=124 IP Failover Object DLL Establish Linkage
10:19:39.156 tid=124 Loading failover object dll fmoracle.dll...
10:19:39.250 tid=124 Oracle Failover Object DLL Process Attach
10:19:39.296 tid=124 Oracle Failover Object DLL Establish Linkage
10:19:39.343 tid=124 Loading failover object dll fmscript.dll...
10:19:39.437 tid=124 Script Failover Object DLL Process Attach
10:19:39.500 tid=124 Script Failover Object DLL Establish Linkage
10:19:39.546 tid=124 Loading failover object dll fmsql.dll...
10:19:39.640 tid=124 Sql Failover Object DLL Process Attach
10:19:39.671 tid=124 Sql Failover Object DLL Establish Linkage
10:19:40.203 tid=124 Created GROUP nexus "mark"
10:19:40.312 tid=124 Script DLL Initialize
10:19:40.312 tid=155 Monitoring group "mark"
10:19:40.406 tid=124 IP DLL Initialize
10:19:40.515 tid=124 Disk Failover Object DLL Initialize
10:19:40.578 tid=124 Step Ib: Forcing Scsi disk driver to discover new 
devices
10:19:40.625 tid=124 Step II: Identifying shared devices by probing Dos 
physical drives
10:19:40.671 tid=124 \Device\Harddisk0 is 
scsi address (0, 0, 0, 0) 
product id 'DEC     HSZ20           V30Z'
10:19:40.734 tid=124 \Device\Harddisk1 is 
scsi address (0, 0, 1, 0) 
product id 'DEC     HSZ20           V30Z'
10:19:40.781 tid=124 The driver for \Device\ScsiPort0 is aic78xx
10:19:40.828 tid=124 The initiator id for \Device\ScsiPort0 is 6
10:19:40.890 tid=124 Step IIIa: bus excl. prot. read partition table/sig
10:20:10.453 tid=155 Waited long enough for group "mark" nexus to come up
10:20:10.453 tid=84  Send: node RIGHT is not connected through the nexus-no 
msg
10:20:10.484 tid=65  Listening for data for group "mark"
10:20:10.546 tid=84  Driver processing partition table for 
\\.\PhysicalDrive1
10:20:10.640 tid=84  PhysicalDrive1 has signature 0x4c0e8759
10:20:10.703 tid=84  Driver processing partition table for 
\\.\PhysicalDrive0
10:20:10.750 tid=84  PhysicalDrive0 has signature 0x4be81037
10:20:10.796 tid=124 Step IV: Match physical devices with signatures with 
CFMD objects
10:20:10.843 tid=242 _disk_4c0e8759 (\\.\PhysicalDrive1): Event thread 
waiting
10:20:10.843 tid=244 _disk_4be81037 (\\.\PhysicalDrive0): Event thread 
waiting
10:20:10.843 tid=124 CFMD logging disk is 
_disk_4be81037\1\\_digitalclusterlog
10:20:10.843 tid=243 _disk_4c0e8759 (\\.\PhysicalDrive1): Test I/O thread 
waiting
10:20:10.843 tid=245 _disk_4be81037 (\\.\PhysicalDrive0): Test I/O thread 
waiting
10:20:11.000 tid=124 ShareInitialize
10:20:11.171 tid=124 Sql DLL Initialize
10:20:11.234 tid=124 Reading sqlListHead...
10:20:11.296 tid=124 SQLDLL: Opening SC Manager...
10:20:11.343 tid=124 SQLDLL: Opening SC MS SQL Service...
10:20:11.406 tid=124 SQLDLL: Creating Monitoring Thread...
10:20:11.453 tid=124 SQLDLL: Loading DB-LIB...
10:20:11.625 tid=124 SQLDLL: Initializing DB-LIB...
10:20:11.671 tid=124 SQLDLL: Establishing DB-LIB handlers...
10:20:11.718 tid=124 SQLDLL: Opening a connection to SQL Server...
10:20:11.843 tid=124 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:20:11.890 tid=124 Msg 5701, Level 0, State 2
10:20:11.953 tid=124 Server '', 
10:20:12.000 tid=124 Changed database context to 'master'.
10:20:12.046 tid=124 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:20:12.109 tid=124 SQLDLL: Adding server RIGHT
10:20:12.390 tid=124 SQLDLL: Cleaning possible suspect DBs
10:20:13.078 tid=124 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:20:13.125 tid=124 Msg 0, Level 0, State 1
10:20:13.171 tid=124 Server '', 
10:20:13.218 tid=124 Procedure 'sp_fallback_DEC_clean', 
10:20:13.250 tid=124 Line 277
10:20:13.296 tid=124 Cleaned 0 suspect databases on shared storage
10:20:13.343 tid=124 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:20:13.390 tid=124 SQLDLL: Registering the RPC management interface...
10:20:13.437 tid=124 SQLDLL: Initialize complete. status = 0
10:20:13.484 tid=124 Oracle DLL Initialize
10:20:13.531 tid=124 RegOpenKeyEx failed: 2
10:20:13.593 tid=124 The application to be failed over is not installed.
    File: E:\CluBuild\src\fm\fmcore\typedata.c    Line: 572
10:20:13.687 tid=239 Send: node RIGHT is not connected through the nexus-no 
msg
10:20:13.687 tid=124 The cluster manager is operational on this system.
    File: E:\CluBuild\src\fm\fmcore\fmstartup.c    Line: 367
10:20:13.734 tid=239 Failure status from I WANT GROUP mark sent to RIGHT is 
0x000008ca
10:20:13.859 tid=239 Putting group "mark" Online
10:20:13.921 tid=239 Disk Failover Object DLL _disk_4be81037 goOnline
10:20:13.953 tid=239 Disk PhysicalDrive0 has gone from *OFFLINE* to 
*ONLINE*
10:20:14.000 tid=239 Driver updating partition table for \\.\PhysicalDrive0
10:20:14.046 tid=239 Assign drive letters for disk sig 0x4be81037 part 4
10:20:14.125 tid=239 P: => \Device\Harddisk0\Partition1
10:20:14.171 tid=239 _disk_4be81037 => \Device\Harddisk0\Partition0
10:20:14.375 tid=239 Usage (Mb): total 4075 used 32 (0.80%) free 4043 
(99.20%)
10:20:14.421 tid=239 Cfmd Log Path set to P:\_digitalclusterlog
10:20:14.968 tid=239 Alias for _disk_4be81037 not assigned because could 
not get volume name
10:20:15.000 tid=239 Disk Failover Object DLL _disk_4c0e8759 goOnline
10:20:15.046 tid=239 Disk PhysicalDrive1 has gone from *OFFLINE* to 
*ONLINE*
10:20:15.109 tid=239 Driver updating partition table for \\.\PhysicalDrive1
10:20:15.156 tid=239 Assign drive letters for disk sig 0x4c0e8759 part 4
10:20:15.203 tid=239 Q: => \Device\Harddisk1\Partition1
10:20:15.250 tid=239 _disk_4c0e8759 => \Device\Harddisk1\Partition0
10:20:15.437 tid=239 Usage (Mb): total 4082 used 17 (0.43%) free 4065 
(99.57%)
10:20:15.921 tid=239 Alias for _disk_4c0e8759 not assigned because could 
not get volume name
10:20:16.406 tid=239 The cluster manager has put group mark ON LINE
on this system. Reason: System Startup.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1478
760.2Log Part 2CHEFS::vfr750.reo.dec.com::harrisonbTue Apr 22 1997 09:22198
10:20:16.484 tid=239 Resynchronizing FM with Cfmd
10:25:41.062 tid=242 Disk PhysicalDrive1 has gone from *STALLED* to 
*ONLINE*
10:25:41.109 tid=244 Disk PhysicalDrive0 has gone from *STALLED* to 
*ONLINE*
10:27:53.515 tid=239 Group mark Node RIGHT went from Offline to Online
10:27:53.515 tid=65  Disk PhysicalDrive1 has gone from *ONLINE* to *PAUSED*
10:27:53.625 tid=65  Disk PhysicalDrive0 has gone from *ONLINE* to *PAUSED*
10:27:53.890 tid=65  Disk PhysicalDrive1 has gone from *PAUSED* to *ONLINE*
10:27:53.937 tid=65  Disk PhysicalDrive0 has gone from *PAUSED* to *ONLINE*
10:27:57.515 tid=65  Putting group "mark" Offline
10:27:57.562 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOffline
10:27:57.609 tid=65  Dismount/controlled (voluntary failover) in progress
10:27:57.671 tid=65  Disk PhysicalDrive1 was success dismounted using 
conventional means
10:27:57.718 tid=65  Disk PhysicalDrive1 has gone from *ONLINE* to 
*OFFLINE*
10:27:57.765 tid=65  Disk Failover Object DLL _disk_4be81037 goOffline
10:27:57.812 tid=65  Dismount/controlled (voluntary failover) in progress
10:27:57.859 tid=65  The Cfmd Log Path has been cleared
10:27:57.921 tid=65  Disk PhysicalDrive0 was success dismounted using 
conventional means
10:27:57.968 tid=65  Disk PhysicalDrive0 has gone from *ONLINE* to 
*OFFLINE*
10:27:58.484 tid=65  The cluster manager has put group mark OFF LINE
on this system. Reason: Failback to primary server.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1701
10:28:52.156 tid=270 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:28:52.203 tid=270 Msg 5701, Level 0, State 2
10:28:52.265 tid=270 Server '', 
10:28:52.296 tid=270 Changed database context to 'master'.
10:28:52.343 tid=270 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:29:45.281 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:29:45.328 tid=118 Msg 5701, Level 0, State 2
10:29:45.375 tid=118 Server '', 
10:29:45.421 tid=118 Changed database context to 'master'.
10:29:45.453 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:30:07.171 tid=260 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:30:07.218 tid=260 Msg 5701, Level 0, State 2
10:30:07.265 tid=260 Server '', 
10:30:07.312 tid=260 Changed database context to 'master'.
10:30:07.359 tid=260 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:30:14.515 tid=65  Fm told to start group "mark"
10:30:14.546 tid=65  Putting group "mark" Online
10:30:14.593 tid=65  Disk Failover Object DLL _disk_4be81037 goOnline
10:30:14.640 tid=65  Disk PhysicalDrive0 has gone from *OFFLINE* to 
*ONLINE*
10:30:14.687 tid=65  Driver updating partition table for \\.\PhysicalDrive0
10:30:14.734 tid=65  Assign drive letters for disk sig 0x4be81037 part 4
10:30:14.781 tid=65  P: => \Device\Harddisk0\Partition1
10:30:14.843 tid=65  _disk_4be81037 => \Device\Harddisk0\Partition0
10:30:15.046 tid=65  Usage (Mb): total 4075 used 32 (0.80%) free 4043 
(99.20%)
10:30:15.109 tid=65  Cfmd Log Path set to P:\_digitalclusterlog
10:30:15.578 tid=65  Alias for _disk_4be81037 not assigned because could 
not get volume name
10:30:15.625 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOnline
10:30:15.671 tid=65  Disk PhysicalDrive1 has gone from *OFFLINE* to 
*ONLINE*
10:30:15.718 tid=65  Driver updating partition table for \\.\PhysicalDrive1
10:30:15.765 tid=65  Assign drive letters for disk sig 0x4c0e8759 part 4
10:30:15.812 tid=65  Q: => \Device\Harddisk1\Partition1
10:30:15.859 tid=65  _disk_4c0e8759 => \Device\Harddisk1\Partition0
10:30:16.015 tid=65  Usage (Mb): total 4082 used 17 (0.43%) free 4065 
(99.57%)
10:30:16.531 tid=65  Alias for _disk_4c0e8759 not assigned because could 
not get volume name
10:30:17.046 tid=65  The cluster manager has put group mark ON LINE
on this system. Reason: Administrator request.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1478
10:30:17.125 tid=65  Resynchronizing FM with Cfmd
10:30:32.718 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:30:32.750 tid=118 Msg 5701, Level 0, State 2
10:30:32.796 tid=118 Server '', 
10:30:32.843 tid=118 Changed database context to 'master'.
10:30:32.890 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:31:26.921 tid=65  Putting group "mark" Offline
10:31:26.968 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOffline
10:31:27.015 tid=65  Dismount/controlled (voluntary failover) in progress
10:31:27.062 tid=65  Disk PhysicalDrive1 was success dismounted using 
conventional means
10:31:27.125 tid=65  Disk PhysicalDrive1 has gone from *ONLINE* to 
*OFFLINE*
10:31:27.171 tid=65  Disk Failover Object DLL _disk_4be81037 goOffline
10:31:27.203 tid=65  Dismount/controlled (voluntary failover) in progress
10:31:27.250 tid=65  The Cfmd Log Path has been cleared
10:31:27.328 tid=65  Disk PhysicalDrive0 was success dismounted using 
conventional means
10:31:27.390 tid=65  Disk PhysicalDrive0 has gone from *ONLINE* to 
*OFFLINE*
10:31:27.875 tid=65  The cluster manager has put group mark OFF LINE
on this system. Reason: Administrator request.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1701
10:32:20.890 tid=65  Fm told to start group "mark"
10:32:20.937 tid=65  Putting group "mark" Online
10:32:20.984 tid=65  Disk Failover Object DLL _disk_4be81037 goOnline
10:32:21.015 tid=65  Disk PhysicalDrive0 has gone from *OFFLINE* to 
*ONLINE*
10:32:21.062 tid=65  Driver updating partition table for \\.\PhysicalDrive0
10:32:21.109 tid=65  Assign drive letters for disk sig 0x4be81037 part 4
10:32:21.156 tid=65  P: => \Device\Harddisk0\Partition1
10:32:21.203 tid=65  _disk_4be81037 => \Device\Harddisk0\Partition0
10:32:21.406 tid=65  Usage (Mb): total 4075 used 32 (0.80%) free 4043 
(99.20%)
10:32:21.453 tid=65  Cfmd Log Path set to P:\_digitalclusterlog
10:32:21.968 tid=65  Alias for _disk_4be81037 not assigned because could 
not get volume name
10:32:22.000 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOnline
10:32:22.062 tid=65  Disk PhysicalDrive1 has gone from *OFFLINE* to 
*ONLINE*
10:32:22.109 tid=65  Driver updating partition table for \\.\PhysicalDrive1
10:32:22.156 tid=65  Assign drive letters for disk sig 0x4c0e8759 part 4
10:32:22.203 tid=65  Q: => \Device\Harddisk1\Partition1
10:32:22.250 tid=65  _disk_4c0e8759 => \Device\Harddisk1\Partition0
10:32:22.406 tid=65  Usage (Mb): total 4082 used 17 (0.43%) free 4065 
(99.57%)
10:32:22.906 tid=65  Alias for _disk_4c0e8759 not assigned because could 
not get volume name
10:32:23.406 tid=65  The cluster manager has put group mark ON LINE
on this system. Reason: Administrator request.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1478
10:32:23.484 tid=65  Resynchronizing FM with Cfmd
10:36:27.812 tid=65  Putting group "mark" Offline
10:36:27.859 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOffline
10:36:27.906 tid=65  Dismount/controlled (voluntary failover) in progress
10:36:27.953 tid=65  Disk PhysicalDrive1 was success dismounted using 
conventional means
10:36:28.000 tid=65  Disk PhysicalDrive1 has gone from *ONLINE* to 
*OFFLINE*
10:36:28.046 tid=65  Disk Failover Object DLL _disk_4be81037 goOffline
10:36:28.078 tid=65  Dismount/controlled (voluntary failover) in progress
10:36:28.125 tid=65  The Cfmd Log Path has been cleared
10:36:28.203 tid=65  Disk PhysicalDrive0 was success dismounted using 
conventional means
10:36:28.250 tid=65  Disk PhysicalDrive0 has gone from *ONLINE* to 
*OFFLINE*
10:36:28.750 tid=65  The cluster manager has put group mark OFF LINE
on this system. Reason: Administrator request.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1701
10:38:21.750 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:38:21.796 tid=118 Msg 5701, Level 0, State 2
10:38:21.843 tid=118 Server '', 
10:38:21.890 tid=118 Changed database context to 'master'.
10:38:21.921 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:38:41.000 tid=165 ModifyGroup: changed the following attributes for 
group mark
10:38:41.031 tid=165 	ObjectList
10:38:41.468 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:38:41.515 tid=118 Msg 5701, Level 0, State 2
10:38:41.562 tid=118 Server '', 
10:38:41.609 tid=118 Changed database context to 'master'.
10:38:41.656 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:23.125 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:23.171 tid=118 Msg 5701, Level 0, State 2
10:39:23.218 tid=118 Server '', 
10:39:23.265 tid=118 Changed database context to 'master'.
10:39:23.296 tid=118 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:47.140 tid=65  Fm told to start group "mark"
10:39:47.187 tid=65  Putting group "mark" Online
10:39:47.218 tid=65  Disk Failover Object DLL _disk_4be81037 goOnline
10:39:47.265 tid=65  Disk PhysicalDrive0 has gone from *OFFLINE* to 
*ONLINE*
10:39:47.312 tid=65  Driver updating partition table for \\.\PhysicalDrive0
10:39:47.375 tid=65  Assign drive letters for disk sig 0x4be81037 part 4
10:39:47.437 tid=65  P: => \Device\Harddisk0\Partition1
10:39:47.484 tid=65  _disk_4be81037 => \Device\Harddisk0\Partition0
10:39:47.859 tid=65  Usage (Mb): total 4075 used 82 (2.03%) free 3993 
(97.97%)
10:39:47.921 tid=65  Cfmd Log Path set to P:\_digitalclusterlog
10:39:48.421 tid=65  Alias for _disk_4be81037 not assigned because could 
not get volume name
10:39:48.453 tid=65  Disk Failover Object DLL _disk_4c0e8759 goOnline
10:39:48.500 tid=65  Disk PhysicalDrive1 has gone from *OFFLINE* to 
*ONLINE*
10:39:48.546 tid=65  Driver updating partition table for \\.\PhysicalDrive1
10:39:48.609 tid=65  Assign drive letters for disk sig 0x4c0e8759 part 4
10:39:48.656 tid=65  Q: => \Device\Harddisk1\Partition1
10:39:48.703 tid=65  _disk_4c0e8759 => \Device\Harddisk1\Partition0
10:39:49.031 tid=65  Usage (Mb): total 4082 used 17 (0.43%) free 4065 
(99.57%)
10:39:49.531 tid=65  Alias for _disk_4c0e8759 not assigned because could 
not get volume name
10:39:49.578 tid=65  Sql DLL GoOnline object mark
760.3Log Part 3 of 3CHEFS::vfr750.reo.dec.com::harrisonbTue Apr 22 1997 09:2279
10:39:49.640 tid=65  SQLDLL: Opening a connection to SQL Server...
10:39:49.703 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:49.750 tid=65  Msg 5701, Level 0, State 2
10:39:49.812 tid=65  Server '', 
10:39:49.859 tid=65  Changed database context to 'master'.
10:39:49.921 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:49.968 tid=65  SQLDLL: Activating DBs...
10:39:50.031 tid=65  SQLDLL: Remap drive(RIGHT, P:, P:)
10:39:50.171 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:50.234 tid=65  Msg 15372, Level 0, State 1
10:39:50.296 tid=65  Server '', 
10:39:50.343 tid=65  Procedure 'sp_fallback_upd_dev_drive', 
10:39:50.390 tid=65  Line 81
10:39:50.453 tid=65  Fallback drive data was updated for 0 devices.
10:39:50.500 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:50.734 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:50.796 tid=65  Msg 15345, Level 11, State 1
10:39:50.859 tid=65  Server '', 
10:39:50.906 tid=65  Procedure 'sp_fallback_activate_svr_db', 
10:39:50.968 tid=65  Line 161
10:39:51.015 tid=65  This fallback server '' has no databases enrolled for 
the primary server 'RIGHT' which match the database name pattern.  Aborting 
this procedure.
10:39:51.078 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:51.125 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:51.187 tid=65  Msg 16916, Level 16, State 1
10:39:51.234 tid=65  Server '', 
10:39:51.296 tid=65  Procedure 'sp_fallback_activate_svr_db', 
10:39:51.359 tid=65  Line 914
10:39:51.406 tid=65  A cursor with the name 'csr_3_db' does not exist
10:39:51.468 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:51.515 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:51.578 tid=65  Msg 16916, Level 16, State 1
10:39:51.640 tid=65  Server '', 
10:39:51.703 tid=65  Procedure 'sp_fallback_activate_svr_db', 
10:39:51.750 tid=65  Line 915
10:39:51.812 tid=65  A cursor with the name 'csr_4_dev' does not exist
10:39:51.859 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:51.921 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:51.968 tid=65  Msg 16916, Level 16, State 1
10:39:52.031 tid=65  Server '', 
10:39:52.078 tid=65  Procedure 'sp_fallback_activate_svr_db', 
10:39:52.140 tid=65  Line 916
10:39:52.203 tid=65  A cursor with the name 'csr_5_usg' does not exist
10:39:52.265 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:52.312 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:52.375 tid=65  Msg 16916, Level 16, State 1
10:39:52.437 tid=65  Server '', 
10:39:52.484 tid=65  Procedure 'sp_fallback_activate_svr_db', 
10:39:52.531 tid=65  Line 917
10:39:52.593 tid=65  A cursor with the name 'csr_6_online' does not exist
10:39:52.640 tid=65  ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
10:39:52.703 tid=65  SQLDLL: dbsqlok failed
10:39:53.265 tid=65  The cluster manager has put group mark ON LINE
on this system. Reason: Administrator request.
    File: E:\CluBuild\src\fm\fmcore\fmgroup.c    Line: 1478
10:39:53.359 tid=65  Resynchronizing FM with Cfmd
10:39:53.921 tid=104 ------------ Sql Failover Object DB-Lib Message 
Handler invoked -------------
10:39:53.968 tid=104 Msg 5701, Level 0, State 2
10:39:54.015 tid=104 Server '', 
10:39:54.078 tid=104 Changed database context to 'master'.
10:39:54.125 tid=104 ------------ Sql Failover Object DB-Lib Message 
Handler ended -------------
760.4sysservers lost your server name?DECWET::CAPPELLOFMy other brain is a polymerWed Apr 23 1997 14:2110
    In the log shown in .3, there are several lines that look like this:
    
10:39:49.812 tid=65  Server '', 
    
    
    That means the server doesn't know its own name.  (That is, the
    servername doesn't appear in the SQL sysservers table.)  I don't know
    why that happened, but it will prevent failover from working correctly. 
    You may have to look at the sp_addserver stored procedure to repair the
    sysservers table.
760.5SUTRA::16.36.2.155::BatsSpeeding, speeding, I'm always speedingThu Apr 24 1997 10:2612

	I bet what happened is that someone reinstalled SQL Server
	on at least of the machines, basically whiping out 
	master.dat.
	Or someone was even more optimistic and executed some
	of the sp_drop_server (or something like that) commands,
	which would destroy your systables entries.
	You'll need quite a SQL guru that straighten that out.
	(For both situations)
	
	Pjotrr