Singularity with grid engine - Revision history

PeterThorpe at 09:21, 21 January 2022

2022-01-21T09:21:45Z

2019-05-28T13:26:48Z

2019-05-28T13:26:19Z

2019-05-28T11:58:53Z

2019-05-24T11:08:44Z

2019-05-24T11:08:13Z

2019-05-24T11:07:09Z

2019-05-24T11:04:02Z

2019-05-24T11:01:36Z

2019-05-24T11:00:46Z

← Older revision		Revision as of 09:21, 21 January 2022
Line 16:		Line 16:
	singularity pull docker:name_of_image		singularity pull docker:name_of_image
	e.g. singularity pull docker:nextgenusfs/funannotate		e.g. singularity pull docker:nextgenusfs/funannotate
		+
		+	Be slightly careful about versions. Teh command may not actually pull down the laters see this below, also how to do this on Rocky Linux 8:
		+	My download command (which pulled down an old version):
		+	singularity pull docker:docker broadinstitute/gatk (did not pull the latest - check versions)
		+
		+	what did pull down the correct version (Redhat 6.9, running on centos 7):
		+
		+	singularity pull docker:broadinstitute/gatk:latest
		+
		+	Now trying on Rocky Linux 8. This worked:
		+
		+	singularity pull docker://broadinstitute/gatk

@@ Line 42: / Line 42: @@
   -pe multi 8     this asks for 8 cores, just as normal.
-please note that n the shell script:
+please note that in the shell script:
   full paths to everything are required!

@@ Line 31: / Line 31: @@
 this is the image you download for the software you are interested in
-   ./test_script.sh
+   /full_path_to/test_script.sh
   this needs to contain the commands you want to run
@@ Line 42: / Line 42: @@
   -pe multi 8     this asks for 8 cores, just as normal.

@@ Line 21: / Line 21: @@
 Example of how to run via qsub:
-  qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif ./test_script.sh
+  qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif /full_path_to/test_script.sh
   replace: ubuntu.sif with whatever image you are trying to run

@@ Line 35: / Line 35: @@
-example 2, runnin the image with qsub:
+example 2, running the image with qsub:
   qsub -pe multi 8 -l singularity -b y singularity run /full_path/funannotate_latest.sif /full_path/fun_singularity.sh
 The shell must have the current working directory full path in it as cd /ful_path/

@@ Line 14: / Line 14: @@
 to download the image, simply type:
-  singularity pull docker:nextgenusfs/funannotate
+ singularity pull docker:name_of_image
 Please, do not run this on the head node, this must be run through the qsub system.

@@ Line 8: / Line 8: @@
   https://www.sylabs.io/guides/3.2/user-guide/
-Example:
+where to get the images from? search here:
   qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif ./test_script.sh
   replace: ubuntu.sif with whatever image you are trying to run
@@ Line 23: / Line 33: @@
-example 2:
+example 2, runnin the image with qsub:
   qsub -pe multi 8 -l singularity -b y singularity run /full_path/funannotate_latest.sif /full_path/fun_singularity.sh
 The shell must have the current working directory full path in it as cd /ful_path/
@@ Line 29: / Line 39: @@
   cd /ful_path/
   -pe multi 8     this asks for 8 cores, just as normal.
 To add another node with singularity on:
+ qconf -me <nodename>
 On the complex_values line remove NONE if present, and add "singularity=TRUE"
 Followed guide here: https://blogs.univa.com/2019/01/using-univa-grid-engine-with-singularity/
 now a request-able resource with "-l singularity" to make sure you get a node with singularity on

@@ Line 5: / Line 5: @@
 Please remember, you must run this through qsub and not directly on the head node.
 Full singularity documentation is here;
+ https://www.sylabs.io/guides/3.2/user-guide/
 Example:
@@ Line 54: / Line 49: @@
 On the complex_values line remove NONE if present, and add "singularity=TRUE"

@@ Line 1: / Line 1: @@
-JW may 2019
+We have been working hard in the background to allow users to be able to run Docker images.
 now a request-able resource with "-l singularity" to make sure you get a node with singularity on
-Full singularity documentation is here; https://blogs.univa.com/2019/01/using-univa-grid-engine-with-singularity/
+Full singularity documentation is here; https://www.sylabs.io/guides/3.2/user-guide/
 Example:
@@ Line 32: / Line 34: @@
   cd /ful_path/
   -pe multi 8     this asks for 8 cores, just as normal.