Difference between revisions of "Frontend Restart"
(Created page with "= Introduction = Contains notes on how to restart marvin. = Provisos = Restarting marvin is a major operation, as all running jobs are lost. It is therefore necessary to a...") |
|||
| Line 2: | Line 2: | ||
Contains notes on how to restart marvin. | Contains notes on how to restart marvin. | ||
| + | |||
| + | = Measures = | ||
| + | |||
| + | == Bring all nodes down before restart == | ||
| + | |||
| + | This is possibly the most useful measure. Primarily, it is due to the nodes using marvin to keep various filesystems mounted, and the havoc they experience when marvin stops doing this. NFS4 stale filehandles then appear and are hard to get rid of. This measure is not immediately obvious, becuase all the nodes are updated on a rolling basis and often do not need to be turned off. And then, when marvin is back up, and once its filesystems are verified, the nodes maybe brought back up. | ||
| + | |||
| + | |||
= Provisos = | = Provisos = | ||
Revision as of 12:29, 1 August 2016
Introduction
Contains notes on how to restart marvin.
Measures
Bring all nodes down before restart
This is possibly the most useful measure. Primarily, it is due to the nodes using marvin to keep various filesystems mounted, and the havoc they experience when marvin stops doing this. NFS4 stale filehandles then appear and are hard to get rid of. This measure is not immediately obvious, becuase all the nodes are updated on a rolling basis and often do not need to be turned off. And then, when marvin is back up, and once its filesystems are verified, the nodes maybe brought back up.
Provisos
Restarting marvin is a major operation, as all running jobs are lost.
It is therefore necessary to advise all users well in advance, as to when it might happen.
