Python DRMAA - Revision history

Rf at 12:13, 21 December 2016

2016-12-21T12:13:27Z

Rf at 11:55, 21 December 2016

2016-12-21T11:55:41Z

Rf: Created page with "=Introduction= A python interface that uses the DRMAA C-binding API to automate parts of running queue jobs, to make the process a bit more mangeable. It is not terribly sop..."

2016-12-05T19:15:07Z

Created page with "=Introduction= A python interface that uses the DRMAA C-binding API to automate parts of running queue jobs, to make the process a bit more mangeable. It is not terribly sop..."

New page

=Introduction=

A python interface that uses the DRMAA C-binding API to automate parts of running queue jobs, to make the process a bit more mangeable.

It is not terribly sophisticated, so is not capable of miracles, though one or two of its abilities may come in handy.

= Key insight =

When a jobTemplate is defined, the queue options under which the job will be run are put into an opaque string, similar to how you would enter the details to qsub on the command line. You set it like this:

s = drmaa.Session()
s.initialize()
jt = s.createJobTemplate()
jt.nativeSpecification='-q single.q -pe multi 2'

So this will run the script on the single.q queue with two threads per script.

So the script can be a normal bash without any pragma (i.e. #$ ) options for the script.

← Older revision		Revision as of 12:13, 21 December 2016
Line 28:		Line 28:

	So the script can be a normal bash without any pragma (i.e. #$ ) options for the script.		So the script can be a normal bash without any pragma (i.e. #$ ) options for the script.
		+
		+	== individual control of running scripts ==
		+
		+	The SGE_TASK_ID variable is typically the only way to control what each individual script in the job array will work on. However it also possible to use the argument list which is passed to each script to obtain further controls. This is not entirely robust, as the arguments become a shell characteristic and subject ot the shell limitations (primarily the size, whihc is something in the 10000's ... in parallel jobs it is easy to exceed this. However, right now it is the only solution without writing to disk.

@@ Line 5: / Line 5: @@
 It is not terribly sophisticated, so is not capable of miracles, though one or two of its abilities may come in handy.
-= Key insight =
+= Key Points =
 When a jobTemplate is defined, the queue options under which the job will be run are put into an opaque string, similar to how you would enter the details to qsub on the command line. You set it like this: