Rf at 13:25, 5 May 2017

2017-05-05T13:25:15Z

Rf at 13:23, 5 May 2017

2017-05-05T13:23:58Z

Rf: Created page with "= Usage of Cluster= * Cluster Manual * Why a Queue Manager? * Available Software = Documented Programs = The following can be seen as extra notes referring to th..."

2017-05-05T12:36:31Z

Created page with "= Usage of Cluster= * Cluster Manual * Why a Queue Manager? * Available Software = Documented Programs = The following can be seen as extra notes referring to th..."

New page

= Usage of Cluster=
* [[Cluster Manual]]
* [[Why a Queue Manager?]]
* [[Available Software]]

= Documented Programs =

The following can be seen as extra notes referring to these programs usage on the marvin cluster, with an emphais on example use-cases. Most, if not all, will have their own special websites, with more detailed manuals and further information.

{|style="width:85%"
|* [[abacas]]
|* [[albacore]]
|* [[ariba]]
|* [[assembly-stats]]
|* [[augustus]]
|* [[BamQC]]
|-
|* [[bcftools]]
|* [[bedtools]]
|* [[BLAST]]
|* [[blast2go: b2g4pipe]]
|* [[bowtie]]
|* [[bowtie2]]
|-
|* [[bwa]]
|* [[BUSCO]]
|* [[CAFE]]
|* [[canu]]
|* [[cd-hit]]
|* [[cegma]]
|-
|* [[clustal]]
|* [[cramtools]]
|* [[detonate]]
|* [[diamond]]
|* [[ea-utils]]
|* [[ensembl]]
|-
|* [[ETE]]
|* [[FASTQC and MultiQC]]
|* [[Archaeopteryx and Forester]]
|* [[GapFiller]]
|* [[GenomeTools]]
|* [[gubbins]]
|-
|* [[JBrowse]]
|* [[kallisto]]
|* [[last]]
|* [[lastz]]
|* [[Mash]]
|* [[mega]]
|-
|* [[MUMmer]]
|* [[NanoSim]]
|* [[OrthoFinder]]
|* [[quast]]
|* [[PGAP]]
|* [[picard-tools]]
|-
|* [[poRe]]
|* [[poretools]]
|* [[prokka]]
|* [[pyrad]]
|* [[python]]
|* [[qualimap]]
|-
|* [[R]]
|* [[RAxML]]
|* [[Repeatmasker]]
|* [[rnammer]]
|* [[roary]]
|* [[RSeQC]]
|-
|* [[samtools]]
|* [[Satsuma]]
|* [[sickle]]
|* [[SPAdes]]
|* [[sra-tools]]
|* [[srst2]]
|-
|* [[SSPACE]]
|* [[stacks]]
|* [[trimmomatic]]
|* [[Trinity]]
|* [[t-coffee]]
|* [[velvet]]
|}

= Queue Manager Tips =
A cluster is a shared resource with different users running different types of analyses. Nearly all clusters use a piece of software called a queue manager to fairly share out the resource. The queue manager on marvin is called Grid Engine, and it has several commands available, all beginning with '''q''' and with '''qsub''' being the most commonly used as it submits a command via a jobscript to be processed. Here are some tips:
* [[Queue Manager Tips]]
* [[General Command-line Tips]]
* [[DRMAA for further Gridengine automation]]

= Data Examples =
* [[Two Eel Scaffolds]]

= Procedures =
(short sequence of tasks with a certain short-term goal, often, a simple script)
* [[Calculating coverage]]
* [[MinION Coverage sensitivity analysis]]

= Navigating genomic data websites=
* [[Patric]]
* [[NCBI]]

= Explanations=
* [[ITUcourse]]
* [[VCF]]

= Pipelines =
(Workflow with a specific end-goal)
* [[Trinity_Protocol]]
* [[STAR BEAST]]
* [[callSNPs.py]]
* [[mapping.py]]
* [[Edgen RNAseq]]
* [[Miseq Prokaryote FASTQ analysis]]
* [[snpcallphylo]]
* [[Bottlenose dolphin population genomic analysis]]

=Protocols=
(Extensive workflows with different with several possible end goals)
* [[Synthetic Long reads]]
* [[MinION (Oxford Nanopore)]]
* [[MinKNOW folders and log files]]

= Cluster Administration =
* [[Hardware Issues]]
* [[Admin Tips]]
* [[RedHat]]
* [[Globus_gridftp]]
* [[Galaxy Setup]]
* [[Son of Gridengine]]
* [[Blas Libraries]]
* [[CMake]]
* [[Users and Groups]]
* [[emailing]]
* [[biotime machine]]
* [[node1 issues]]
* [[6TB storage expansion]]
* [[Home directories max-out incident 28.11.2016]]
* [[Frontend Restart]]
* [[environment-modules]]
* [[H: drive on cluster]]
* [[Incident: Can't connect to BerkeleyDB]]
* [[Bioinformatics Wordpress Site]]
* [[Backups]]
* [[Python DRMAA]]
* [[SAN disconnect incident 10.01.2017]]
* [[Memory repair glitch 16.02.2017]]
* [[node9 network failure incident 16-20.03.2017]]

= Courses =

==I2U4BGA==
* [[Original schedule]]
* [[New schedule]]
* [[Actual schedule]]
* [[Course itself]]
* [[Biolinux Source course]]
* [[Directory Organization Exercise]]
* [[Glossary]]
* [[Key Bindings]]
* [[one-liners]]
* [[Cheatsheets]]
* [[Links]]
* [[pandoc modified manual]]
* [[Command Line Exercises]]

= hdi2u =

The half-day linux course held on 20th April. Modified version of I2U4BGA.

* [[hdi2u_intro]]
* [[hdi2u_commandbased_exercises]]
* [[hdi2u_dirorg_exercise]]
* [[hdi2u_rendertotsv_exercise]]

= RNAseq for DGE =
* [[Theoretical background]]
* [[Quality Control and Preprocessing]]
* [[Mapping to Reference]]
* [[Mapping Quality Exercise]]
* [[Key Aspects of using R]]
* [[Estimating Gene Count Exercise]]
* [[Differential Expression Exercise]]
* [[Functional Analysis Exercise]]

==Templates==
* [[edgenl2g]]

@@ Line 96: / Line 96: @@
 Using process substitution, or named pipes, you can clean multiple fastq's in one pass. This is useful for combining multiple MiSeq runs, or multiple lanes for example:
-fastq-mcf -o cleaned.R1.fq.gz -o cleaned.R2.fq.gz adapters.fa \
+ fastq-mcf -o cleaned.R1.fq.gz -o cleaned.R2.fq.gz adapters.fa \
    <(gunzip -c uncleaned.lane1.R1.fq.gz uncleaned.lane2.R1.fq.gz;) \
    <(gunzip -c uncleaned.lane1.R2.fq.gz uncleaned.lane2.R2.fq.gz;)

Ea-utils - Revision history

Rf at 13:25, 5 May 2017

Rf at 13:23, 5 May 2017

Rf: Created page with "= Usage of Cluster= * Cluster Manual * Why a Queue Manager? * Available Software = Documented Programs = The following can be seen as extra notes referring to th..."