Difference between revisions of "Singularity with grid engine"

From wiki
Jump to: navigation, search
Line 7: Line 7:
  
  
−
now a requestable resource with "-l singularity" to make sure you get a node with singularity on
+
now a request-able resource with "-l singularity" to make sure you get a node with singularity on
  
  
 
Example:
 
Example:
 
  qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif ./test_script.sh
 
  qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif ./test_script.sh
−
  replace:ubuntu.sif with whatever image you are trying to run
+
  replace: ubuntu.sif with whatever image you are trying to run
  
 +
Lets go through that command in more depth:
 +
qsub -l singularity -b y singularity run
  
−
example2:
+
this is a special command so singularity will run on a specific server
 +
  /full_path_to/ubuntu.sif
 +
this is the image you download for the software you are interested in
 +
 
 +
  ./test_script.sh
 +
this needs to contain the commands you want to run
 +
 
 +
 
 +
example 2:
 
  qsub -pe multi 8 -l singularity -b y singularity run /full_path/funannotate_latest.sif /full_path/fun_singularity.sh
 
  qsub -pe multi 8 -l singularity -b y singularity run /full_path/funannotate_latest.sif /full_path/fun_singularity.sh
−
The shell must have the curent working directory full path in it as cd /ful_path/   
+
The shell must have the current working directory full path in it as cd /ful_path/   
−
  the #!cwd command will not work!
+
  putting #!cwd command in your shell scripts will not work!
 +
cd /ful_path/
 +
-pe multi 8    this asks for 8 cores, just as normal.
  
  
Line 27: Line 39:
  
 
On the complex_values line remove NONE if present, and add "singularity=TRUE"
 
On the complex_values line remove NONE if present, and add "singularity=TRUE"
−
 
−
 
−
 
−
 
−
-b is a normal qsub command
 
−
 
−
      -b y[es]|n[o]
 
−
              Available for qsub, qrsh only. Qalter does not allow changing this option. This  option  cannot
 
−
              be embedded in the script file itself.
 
−
 
−
              Gives  the  user  the  possibility  to indicate explicitly whether command should be treated as
 
−
              binary or script. If the argument of -b is ’y’, then command  may be a binary or  script.  The
 
−
              command  might not be accessible from the submission host.  Nothing except the path of the com-
 
−
              mand will be transferred from the submission host to the execution host. Path aliasing will  be
 
−
              applied to the path of command before it is executed.
 
−
 
−
              If the argument of -b is ’n’ then command needs to be a script, and it will be handled as such.
 
−
              The script file has to be accessible by the submission host. It will be transferred to the exe-
 
−
              cution host. qsub/qrsh will search directive prefixes within scripts.
 
−
 
−
              qsub will implicitly use -b n, whereas qrsh will apply the -b y option if nothing else is spec-
 
−
              ified.
 
−
 
−
              The value specified with this option or the corresponding value specified in qmon will only  be
 
−
              passed  to defined JSV instances if the value is yes.  The name of the parameter will be b. The
 
−
              value will be y also when the long form yes was specified during submission.  (See -jsv  option
 
−
              below or find more information concerning JSV in jsv(1).)
 
−
 
−
              Please  note  that  submission of command as a script (-b n) can have a significant performance
 
−
              impact, especially for short running jobs and big job scripts.  Script submission adds a number
 
−
              of operations to the submission process: The job script needs to be
 
−
              - parsed at client side (for special comments)
 
−
              - transferred from submit client to qmaster
 
−
              - spooled in qmaster
 
−
              - transferred to execd at job execution
 
−
              - spooled in execd
 
−
              - removed from spooling both in execd and qmaster once the job is done
 
−
              If job scripts are available on the execution nodes, e.g. via NFS, binary submission can be the
 
−
              better choice.
 

Revision as of 10:40, 24 May 2019

JW may 2019


Followed guide here: https://blogs.univa.com/2019/01/using-univa-grid-engine-with-singularity/

Pete installed singularity on phylo


now a request-able resource with "-l singularity" to make sure you get a node with singularity on


Example: 

qsub -l singularity -b y singularity run /full_path_to/ubuntu.sif ./test_script.sh
replace: ubuntu.sif with whatever image you are trying to run

Lets go through that command in more depth: 

qsub -l singularity -b y singularity run

this is a special command so singularity will run on a specific server 

 /full_path_to/ubuntu.sif

this is the image you download for the software you are interested in 

 ./test_script.sh
this needs to contain the commands you want to run


example 2: 

qsub -pe multi 8 -l singularity -b y singularity run /full_path/funannotate_latest.sif /full_path/fun_singularity.sh

The shell must have the current working directory full path in it as cd /ful_path/ 

putting #!cwd command in your shell scripts will not work!
cd /ful_path/
-pe multi 8     this asks for 8 cores, just as normal.


To add another node with singularity on:


qconf -me <nodename>

On the complex_values line remove NONE if present, and add "singularity=TRUE"

Retrieved from "http://stab.st-andrews.ac.uk/wiki/index.php?title=Singularity_with_grid_engine&oldid=3400"