http://stab.st-andrews.ac.uk/wiki/index.php?title=Special:NewPages&feed=atom&hideredirs=1&limit=50&offset=&namespace=0&username=&tagfilter=&size-mode=max&size=0wiki - New pages [en]2024-03-29T02:08:41ZFrom wikiMediaWiki 1.30.0http://stab.st-andrews.ac.uk/wiki/index.php/Firewall_and_iptablesFirewall and iptables2020-07-22T18:21:05Z<p>PeterThorpe: Created page with "firstly the iptables was accidentally wiped when trying to mount marvin to kennedy. SGE failed and so did ldap authentication from the nodes on new users. solution for ldap..."</p>
<hr />
<div>firstly the iptables was accidentally wiped when trying to mount marvin to kennedy. SGE failed and so did ldap authentication from the nodes on new users. <br />
<br />
solution for ldap<br />
Fixed. Ldap is not ldap. It is smbladp which listens on port 1544. OPen up port 1544 (see below on how to open up ports)<br />
<br />
restart ldap<br />
service slapd restart<br />
<br />
How to turn on and off firwll and save to iptables. <br />
https://www.cyberciti.biz/faq/turn-on-turn-off-firewall-in-linux/<br />
<br />
/etc/init.d/iptables save<br />
/etc/init.d/iptables stop<br />
<br />
<br />
SGE stopped working: <br />
Got it working was on port 6444<br />
<br />
To open firewall ports. Log in as root, but ssh as root with -x<br />
system-config-firewall <br />
<br />
"other ports" on the left - open the bad boys you want open.</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Disk_management_after_shelf_disk_failureDisk management after shelf disk failure2020-06-19T11:15:11Z<p>PeterThorpe: Created page with " The software for your RAID controller was LSI (now Broadcom) MegaCli, I've not been able to find any evidence that there's a Redhat package but I have found a download. The..."</p>
<hr />
<div><br />
<br />
The software for your RAID controller was LSI (now Broadcom) MegaCli, I've not been able to find any evidence that there's a Redhat package but I have found a download. The URL below should get you a zip file that contains a rpm and other installers for other operating systems.<br />
<br />
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.broadcom.com%2Fdocs%2F12351587&amp;data=02%7C01%7Cjrc9%40st-andrews.ac.uk%7Cbe3c1405b577499c6ef808d813989466%7Cf85626cb0da849d3aa5864ef678ef01a%7C0%7C0%7C637280893338362007&amp;sdata=p93NQ8JyhxGgqCHMiAZEHq8xSdODq1urak0LKSNo3%2F0%3D&amp;reserved=0<br />
<br />
If you can install the RPM that you should be able to display the current status of the array to confirm it's safe to remove the failed or failing disk and if necessary configure the new disks. The following commands should show the status of any RAID volumes and physical disks, if you can pipe the output into a couple of files and send it over I can advise on what needs to be done.<br />
<br />
/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aall<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aall<br />
<br />
The first command should produce something like this, we'll be able to tell if the array is degraded or not.<br />
<br />
> Adapter 0 -- Virtual Drive Information:<br />
> Virtual Drive: 0 (Target Id: 0)<br />
> Name :<br />
> RAID Level : Primary-1, Secondary-3, RAID Level Qualifier-0<br />
> Size : 203.25 GB<br />
> Sector Size : 512<br />
> Mirror Data : 203.25 GB<br />
> State : Optimal<br />
> Strip Size : 64 KB<br />
> Number Of Drives per span:2<br />
> Span Depth : 3<br />
> Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache <br />
> if Bad BBU Current Cache Policy: WriteBack, ReadAdaptive, Direct, No <br />
> Write Cache if Bad BBU Default Access Policy: Read/Write Current <br />
> Access Policy: Read/Write<br />
> Disk Cache Policy : Disk's Default<br />
> Encryption Type : None<br />
> Is VD Cached: No<br />
<br />
The second command will produce a lot of output for each disk. The 'Firmware State' line should show how each disk is configured, most should be 'Online' or 'Hotspare' . Once the new disks are added we'll need to re-run this, they'll probably be listed as 'Unconfigured (Good)'. With some information about the disk position the command to configure the disks will be something like the following, I'll confirm when we know what the variable are.<br />
<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDHSP -PhysDrv\[$ENCLOSURE:$SLOT\] -a$ARRAY<br />
<br />
Then if the array is degraded it should automatically start to rebuild using one or more of the new disks.<br />
<br />
If it's still rebuilding this should show the progress:<br />
<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ShowProg -PhysDrv\[8:13\] -a0<br />
<br />
I added one additional disk to each tray, they should be in Enclosure 8 Slot 14 and Enclosure 9 Slot 13. Confirm that with the output from PDList, if that's correct this should mark them as hotspares.<br />
<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDHSP -Set -PhysDrv\[8:14\] -a0<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDHSP - Set -PhysDrv\[9:13\] -a0<br />
<br />
Confirm by running PDList again, their Firmware State should have updated from Unconfigured Good to Configured Hotspare or something similar.<br />
<br />
<br />
to set as hot spares<br />
<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDHSP -Set -PhysDrv\[8:14\] -a0<br />
/opt/MegaRAID/MegaCli/MegaCli64 -PDHSP -Set -PhysDrv\[9:13\] -a0<br />
<br />
<br />
From the verbose MegaCli -h … all the options in some kind of order<br />
> MegaCli -PDHSP {-Set [-Dedicated [-ArrayN|-Array0,1,2...]] [-EnclAffinity] [-nonRevertible]}<br />
> |-Rmv -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Samba_like_connectionSamba like connection2020-05-06T12:30:11Z<p>PeterThorpe: Created page with " == Windows net work connect in a samba like manner == http://stab.st-andrews.ac.uk/wiki/index.php/Windows_network_connect You probably don't know what samba is. This does..."</p>
<hr />
<div><br />
<br />
== Windows net work connect in a samba like manner ==<br />
<br />
http://stab.st-andrews.ac.uk/wiki/index.php/Windows_network_connect<br />
<br />
You probably don't know what samba is. This doesn't matter. ALl you need to know is - THIS IS COOL.<br />
<br />
install these on your windows or Linux PC. These are prebuilt installer and all you need to do it double click.<br />
sshfs-win: https://github.com/billziss-gh/sshfs-win <br />
Requires WinFsp (https://github.com/billziss-gh/winfsp/releases/latest ) <br />
# to be installed with Cygwin FUSE support ticked in the installer.<br />
<br />
Now open Windows Explorer and go to “This PC”.<br />
From the ribbon click “Map Network Drive”. Map this to a letter of your choice, e.g. k for kennedy<br />
Enter:<br />
<br />
\\sshfs.k\you@kennedy10<br />
<br />
Large file transfers, please use filezilla, I dont know how stable this is!!!!!!!! <br />
<br />
<br />
==Mount the scratch, you still have to navigate to it, like so:==<br />
\\sshfs.k\USERNAME@kennedy10/../../scratch/bioinf/USERNAME<br />
<br />
<br />
To map the scratch see the next slide.</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Submit_a_job_and_monitor_queuesSubmit a job and monitor queues2020-05-06T12:27:43Z<p>PeterThorpe: Created page with " == submit a job == to submit a shell which contains are the needed flags in the file. sbatch spades.sh ==Show information on the queues:== smap squeue (show the queue..."</p>
<hr />
<div><br />
== submit a job ==<br />
<br />
to submit a shell which contains are the needed flags in the file.<br />
sbatch spades.sh<br />
<br />
<br />
==Show information on the queues:==<br />
smap<br />
squeue (show the queue)<br />
squeue –p bigmem (show the queue for big mem)<br />
sview<br />
<br />
==Interactive mode (like qrsh):==<br />
srun --pty bash -p bigmem (Bioinf user : use the bigmem q)</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Filezilla_data_transferFilezilla data transfer2020-05-06T12:20:24Z<p>PeterThorpe: Created page with "Media:file:///C:/Users/pjt6/Desktop/Picture1.png"</p>
<hr />
<div>[[Media:file:///C:/Users/pjt6/Desktop/Picture1.png]]</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Pull_and_push_to_to_and_from_MARVINPull and push to to and from MARVIN2020-05-06T12:17:36Z<p>PeterThorpe: Created page with " == Copy data from marvin – or push to marvin for backup == users will want to copy data from the backed up area Marvin to the temp work/ scratch space on Kennedy, and visa..."</p>
<hr />
<div><br />
== Copy data from marvin – or push to marvin for backup ==<br />
<br />
users will want to copy data from the backed up area Marvin to the temp work/ scratch space on Kennedy, and visa versa. <br />
We recommend using rsync for this. It is surprisingly fast. <br />
<br />
https://www.tecmint.com/rsync-local-remote-file-synchronization-commands/<br />
<br />
rsync -av $USER@marvin.st-andrews.ac.uk:path/ path/<br />
<br />
Type yes, then your marvin password<br />
<br />
PUSH FILES TO MARVIN (please look up the commands)<br />
<br />
rsync -avzhe ssh file_to_transfer $USER@marvin.st-andrews.ac.uk:/path_to/</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Mobaxterm_for_WindowsMobaxterm for Windows2020-05-06T12:13:33Z<p>PeterThorpe: </p>
<hr />
<div>If you use previously use MOBA and want to continue to use it. This is how. You can use any ssh client you wish<br />
<br />
NOTE: for myself, this was done before we changed the way people log in. This may not work for you until we test it. <br />
<br />
On mobaxterm: ( you may need to change the permission of the ssh key folder). <br />
<br />
ssh -i path_to_key username@kennedy.st-andrews.ac.uk<br />
<br />
On windows, mobaxterm required: <br />
<br />
/drives/c/ ….<br />
<br />
It is possible to save this session .. Google is your friend.</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Loggin_on_to_KennedyLoggin on to Kennedy2020-05-06T12:11:25Z<p>PeterThorpe: </p>
<hr />
<div><br />
== Log onto Kennedy using a terminal e.g. putty for Windows or terminal for Linux/ MAC ==<br />
<br />
*[[Mobaxterm for Windows]]<br />
<br />
When you have received a password from the sys admins. <br />
<br />
You should be able to log onto kennedy from a terminal by doing: see next slide.<br />
<br />
ssh USERNAME@kennedy.st-Andrews.ac.uk<br />
<br />
PASSWORD:<br />
<br />
You will then be prompted for your password. <br />
<br />
Please change your password on 1st log in conforming to the university requirements: <br />
https://www.st-andrews.ac.uk/it-support/security/password/<br />
<br />
A strong password is around 10 characters, including uppercase and lowercase letters, numbers and symbols.<br />
<br />
Use the command:<br />
passwd</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Windows_usersWindows users2020-05-06T12:08:24Z<p>PeterThorpe: Created page with " == Windows: Create a public and private key using putty-gen == Type putty gen in the window search bar (download and install if needed) Click on generate and wiggle the mo..."</p>
<hr />
<div><br />
== Windows: Create a public and private key using putty-gen ==<br />
<br />
<br />
Type putty gen in the window search bar (download and install if needed)<br />
<br />
Click on generate and wiggle the mouse. <br />
<br />
Save private key to : <br />
C:\Users\username\.ssh\putty_priv.ppk<br />
<br />
(do not ever share you private key)<br />
<br />
Send the string in the upper grey part of the PuTTYgen window that says <br />
"Public key for pasting into OpenSSH authorized_keys file". <br />
Copy and paste this into a text file or just into the email to the sys admins.<br />
<br />
note for admins:<br />
The key should be called/renamed authorized_keys when put in the .ssh folder in the $HOME on kennedy – this will be done by the sys admin</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Apple_MAC_people_%2B_LinuxApple MAC people + Linux2020-05-06T12:05:35Z<p>PeterThorpe: Created page with " == create an ssh key using MAC or Linux platforms == open a terminal and enter the command "ssh-keygen". Accept all default filenames. Choose a passphrase when asked f..."</p>
<hr />
<div><br />
== create an ssh key using MAC or Linux platforms ==<br />
<br />
<br />
<br />
open a terminal and enter the command <br />
"ssh-keygen". Accept all default filenames. <br />
<br />
Choose a passphrase when asked for one. Then email the sys admins the file <br />
.ssh/id_rsa.pub<br />
<br />
Directories starting with a . are normally not visible, so it might be easiest to first copy that file into your home<br />
directory:<br />
<br />
cp ~/.ssh/id_rsa.pub</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Creating_ssh_keys_and_logging_onCreating ssh keys and logging on2020-05-06T12:03:57Z<p>PeterThorpe: </p>
<hr />
<div><br />
== the security we use requires ssh keys ==<br />
<br />
*[[Apple MAC people + Linux]]<br />
<br />
*[[Windows users]]</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Slurm_commandsSlurm commands2020-05-06T11:56:44Z<p>PeterThorpe: </p>
<hr />
<div><br />
<br />
== slurm commands ==<br />
<br />
note the BIOINF community are supposed to use the -p bigmem queue. <br />
<br />
Sun Grid Engines (what we use on Marvin) to slurm command conversion:<br />
https://srcc.stanford.edu/sge-slurm-conversion<br />
<br />
<br />
<br />
The following are command line commands, or tags you can put in your shell script (or at the command line) to achieve certain functionality.<br />
<br />
request_48_thread_1.3TBRAM<br />
#!/bin/bash -l '''# not the -l is essential now'''<br />
#SBATCH -J fly_pilon #jobname<br />
#SBATCH -N 1 #node<br />
#SBATCH --ntasks-per-node=48<br />
#SBATCH --threads-per-core=2<br />
#SBATCH -p bigmem<br />
#SBATCH --nodelist=kennedy150 # this is the specific node. This one has 1.5TB RAM<br />
#SBATCH --mem=1350GB<br />
<br />
<br />
test_conda_activate<br />
#!/bin/bash -l<br />
#SBATCH -J conda_test #jobname<br />
#SBATCH -N 1 #node<br />
#SBATCH --tasks-per-node=1<br />
#SBATCH -p bigmem # big mem if for the BIOINF community<br />
#SBATCH --mail-type=END # email at the end of the job<br />
#SBATCH --mail-user=$USER@st-andrews.ac.uk # your email address<br />
<br />
cd /gpfs1/home/$USER/<br />
<br />
pyv="$(python -V 2>&1)"<br />
<br />
echo "$pyv"<br />
<br />
# conda to activate the software<br />
<br />
echo $PATH<br />
<br />
conda activate spades<br />
<br />
pyv="$(python -V 2>&1)"<br />
<br />
echo "$pyv"<br />
<br />
conda deactivate <br />
<br />
conda activate python27<br />
<br />
pyv="$(python2 -V 2>&1)"<br />
<br />
echo "$pyv"<br />
<br />
12threads_bigMeme_30G_RAM<br />
<br />
!/bin/bash -l # essential <br />
#SBATCH -J trimmo #jobname<br />
#SBATCH -N 1 #node<br />
#SBATCH --ntasks-per-node=12<br />
#SBATCH --threads-per-core=2<br />
#SBATCH -p bigmem<br />
#SBATCH --mem=30GB<br />
<br />
<br />
Request an interactive job with one GPU:<br />
# srun --gres=gpu:1 -N 1 -p singlenode --pty /bin/bash</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Quick_startQuick start2020-05-06T11:55:20Z<p>PeterThorpe: </p>
<hr />
<div><br />
== quick start ==<br />
<br />
You must either be on the University campus or logged on via a VPN see <br />
https://www.st-andrews.ac.uk/it-support/services/internet/vpn/<br />
<br />
<br />
<br />
Temporary quick start info can be found here. This is a work in progress until something official is made up.<br />
<br />
https://github.com/peterthorpe5/How_to_use_Kennedy_HPC<br />
<br />
<br />
'''Herbert’s excellent existing documentation<br />
'''<br />
Please also see here:<br />
<br />
<br />
https://universityofstandrews907.sharepoint.com/sites/chemistry/CurrentStudents/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2Fchemistry%2FCurrentStudents%2FShared%20Documents%2Fug%2FERCF%5FIntroduction%2Epdf&parent=%2Fsites%2Fchemistry%2FCurrentStudents%2FShared%20Documents%2Fug&p=true&originalPath=aHR0cHM6Ly91bml2ZXJzaXR5b2ZzdGFuZHJld3M5MDcuc2hhcmVwb2ludC5jb20vOmI6L3MvY2hlbWlzdHJ5L0N1cnJlbnRTdHVkZW50cy9FY1RrWjZzZ0xfaEZnZzFGY0dUZXRZNEJicHFCWHhvNF9Pc3cwRXF3NGlXNDFRP3J0aW1lPUEyLVl0YW5jMTBn</div>PeterThorpehttp://stab.st-andrews.ac.uk/wiki/index.php/Kennedy_manualKennedy manual2020-05-06T07:59:32Z<p>PeterThorpe: </p>
<hr />
<div><br />
*[[quick start]]<br />
<br />
#[[creating ssh keys and logging on]]<br />
#[[logging on to Kennedy]]<br />
<br />
*[[pull and push to to and from MARVIN]]<br />
<br />
*[[filezilla data transfer]]<br />
<br />
*[[slurm commands]]<br />
<br />
*[[submit a job and monitor queues]]<br />
<br />
*[[samba like connection]]<br />
<br />
*[[Conda]]<br />
CONDA: http://stab.st-andrews.ac.uk/wiki/index.php/Conda#conda<br />
Training: github.com/peterthorpe5/Sys_admin/tree/master/cluster_course<br />
Note: on Kennedy, you may need to make programs executable to run. This is a know bug. Locate where the install have just gone, move to that directory and run <br />
chmod u+x prog(s)_of_interest. often in th elib folder<br />
<br />
The error that indicates this kind of error is:<br />
libptf77blas.so.3: '''cannot open shared object file<br />
'''<br />
<br />
==syntax highlighting in Nano==<br />
do the following<br />
cp /gpfs1/apps/kennedy_sys_admin/misc/.nanorc ./</div>PeterThorpe