Conference turris::digital_unix

10037.0. "how to dismount NFS when server is gone" by COL01::VSEMUSCHIN (Duck and Recover !) Wed Jun 04 1997 11:32

    This is a cross posting from TruCluster notes conference, but I think
    the topic is as well of general UNIX interest. So ...
    
    The question is how could one dismount NFS, when NFS server hung ?
    
    I have a script (see below) that will shot dead any person pointed out 
    by lsof and then umount -f . But this scripts itself seems to hung. Of
    any killer deserved such an end, but it is Our Son of Bitch. Besides it 
    I see, that some processes (in other decterm windows) that seems to 
    have nothing common with NFS hung too. Please help me.
    
    =Seva
    
    Following is the note 2008 from TruCluster notes conference:
    -------------------------------------------------------------------------
    
    I try to implement following cluster 
    (DUNIX 3.2G/ASE 1.3):
    
    Alpha1 ------- disk1 ------- Alpha2
                   disk2
    
    Alpha1 mounts both disks (HSZ40 RAID sets in SW300) local
    and exports them to Alpha2 via NFS. There are some appli-
    cations and users on Alpha2 that use those disks. There
    are no intentions to export NFS to other computers, we
    try only to share disks among Alpha1 and Alpha2 like in
    TrueClusters (VMSclusters ;-). It was the reason to
    avoid NFS services. 
    
    When Alpha1 goes down, Alpha2 should umount NFS disks and
    mount them to the same mount points locally. But when I
    try to shot processes using those disks, my scrits hung.
    Sometimes I experience messages like 'NFS server doesn't
    answer still trying' even if I at that time use some 
    harmless commands, like man. No, NFS mount points aren't
    referenced in PATH. I use now lsof -S 2, but I already 
    tried fuser-based program (fuser), given me by one.
    colleague. Last one (program ;-) hungs hopelessly.
    
    PLEASE HELP !  I HAVE NO MORE IDEAS WHAT TO DO !
    made by HSZ40 
    
    = Seva
    
    Some quotations fro start/stop scrip follows:
    
    typeset -f DoUmount
    # Comments striped out
    DoUmount() {
    
        echo ${ME} -- Mount list prior to umount >> ${LGFIL}
        mount >> ${LGFIL}
    
        let i=1                                                           
        while [ i -le ${MAX_TRY} ] ; do
                                     
            UsrCnt=$( ${LSOF} -t ${MnPnt} | wc -l)
            echo ${ME} -- ${MnPnt} is used ${UsrCnt} times >> ${LGFIL}
            ${LSOF} ${MnPnt} >> ${LGFIL} 2>&1 
    
    
        if [ ${UsrCnt} -le 0 ] ; then break; fi
     
            for UsrTTY in \
                $(ps -o tname -p "$( ${LSOF} -t ${MnPnt})" | uniq | grep tty)
            do
                echo Leave ${MnPnt}, please  | write - ${UsrTTY}
                echo ${ME} -- Terminal ${UsrTTY} informed >> ${LGFIL}
            done
    
            sleep ${TMO_TRY}
    
            let i=i+1 
        done
    
    
        echo ${ME} -- Following user of ${MnPnt} will be stopped >> ${LGFIL}
        ps -p "$( ${LSOF} -t ${MnPnt})" >> ${LGFIL}
        
        
        while [ $( ${LSOF} -t ${MnPnt} | wc -l ) -gt 0 ] ; do
            echo ${ME} -- Killing $({LSOF} -t ${MnPnt}) >> ${LGFIL}
            kill -KILL $( ${LSOF} -t ${MnPnt}) >> ${LGFIL} 2>&1
            sleep 1
        done
    
        echo ${ME} -- umount [-f] ${MnPnt} >> ${LGFIL}
        if [ ${MnTyp} = nfs ] ; then
            umount -f ${MnPnt}  >> $LGFIL 2>&1
        else
            umount ${MnPnt}  >> $LGFIL 2>&1
        fi
        RetCode=$?
       
    
        if [ ${RetCode} -ne 0 ]; then
            echo ${ME} -- umount failed. Code: ${RetCode}
            exit ${RetCode}
        else
            echo ${ME} -- umount succeed. Code: ${RetCode} >> ${LGFIL
            mount >> ${LGFIL}
        fi
    } # DoUmount
    
    ....
    
    LSOF='/usr/bin/lsof -S 2' 
    
    ....
    
    
    if [ $(mount | grep ${MnPnt} | wc -l) -gt 0 ]; then
        MnDev=$(mount | grep ${MnPnt} | awk '{printf $1}')
        MnTyp=$(mount | grep ${MnPnt} | awk '{printf $5}')
    else 
        MnTyp=dismount
        MnDev=unknown
    fi
    
    # 
    # Lock file is used to let another scripts to wait, until we're
    # ready.
    #
    if [ -a ${ASETMP}/${SrvName}.locked ] ; then 
        rm -f ${ASETMP}/${SrvName}.locked >> ${LGFIL} 2>&1
    fi
    echo ${OpCode} > ${ASETMP}/${SrvName}.locked 
    
    
    case ${OpCode} in
    ( stop )
        case ${MnTyp} in
        ( ufs | advfs ) 
            echo ${ME} -- ${MnPnt} will be dismounted >> ${LGFIL}
            DoUmount
            ;;
        ( nfs | dismount )
            echo ${ME} -- no changes on ${MnPnt} required >> ${LGFIL}
            ;;
        esac
        ;;
    ( start )
        case ${MnTyp} in
        ( nfs )
            echo ${ME} -- ${MnPnt} NFS-dismount\/LOCAL-mount>> ${LGFIL}
            DoUmount
            DoMount
            ;;
        ( ufs | advfs )
            echo ${ME} --  ${MnPnt} is already mounted >> ${LGFIL}
            ;;
    
        ( dismount)
            echo ${ME} -- mounting ${MnPnt}  >> ${LGFIL}
            DoMount
            ;;
        esac
        # RemoteMount   # Written but not used
        ;;
    esac
    
    rm -f ${ASETMP}/${SrvName}.locked >> ${LGFIL} 2>&1
    
    
    

    it seems to be solved.
    it looks like when somebode does cd to directory where nfs volume
    is mounted, AFTER the nfs server death, he cannot be found by both
    lsof and fuser, but umount sees it and tells that file system is
    busy.
    
    Thank you guys, it was a pleasure to ask you for help :-7
    
    =Seva