Difference between revisions of "Main Page"

From wiki
Jump to: navigation, search
(Cluster Administration)
(22 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
* [[Why a Queue Manager?]]
 
* [[Why a Queue Manager?]]
 
* [[Available Software]]
 
* [[Available Software]]
 +
* [[how to use the cluster training course]]
  
 
= Documented Programs =
 
= Documented Programs =
Line 38: Line 39:
 
|-
 
|-
 
|* [[cramtools]]
 
|* [[cramtools]]
 +
|* [[conda]]
 
|* [[deeptools]]
 
|* [[deeptools]]
 
|* [[detonate]]
 
|* [[detonate]]
Line 63: Line 65:
 
|* [[MUMmer]]
 
|* [[MUMmer]]
 
|* [[NanoSim]]
 
|* [[NanoSim]]
 +
|* [[nseq]]
 +
|-
 
|* [[OrthoFinder]]
 
|* [[OrthoFinder]]
|-
+
|* [[PASA]]
 
|* [[perl]]
 
|* [[perl]]
 
|* [[PGAP]]
 
|* [[PGAP]]
 
|* [[picard-tools]]
 
|* [[picard-tools]]
 
|* [[poRe]]
 
|* [[poRe]]
 +
|-
 
|* [[poretools]]
 
|* [[poretools]]
 
|* [[prokka]]
 
|* [[prokka]]
|-
 
 
|* [[pyrad]]
 
|* [[pyrad]]
 
|* [[python]]
 
|* [[python]]
 
|* [[qualimap]]
 
|* [[qualimap]]
 
|* [[quast]]
 
|* [[quast]]
 +
|-
 +
|* [[qiime2]]
 
|* [[R]]
 
|* [[R]]
 
|* [[RAxML]]
 
|* [[RAxML]]
|-
 
 
|* [[Repeatmasker]]
 
|* [[Repeatmasker]]
 +
|* [[Repeatmodeler]]
 
|* [[rnammer]]
 
|* [[rnammer]]
 +
|-
 
|* [[roary]]
 
|* [[roary]]
 
|* [[RSeQC]]
 
|* [[RSeQC]]
 
|* [[samtools]]
 
|* [[samtools]]
 
|* [[Satsuma]]
 
|* [[Satsuma]]
|-
 
 
|* [[sickle]]
 
|* [[sickle]]
 
|* [[SPAdes]]
 
|* [[SPAdes]]
 +
|-
 
|* [[squid]]
 
|* [[squid]]
 
|* [[sra-tools]]
 
|* [[sra-tools]]
 
|* [[srst2]]
 
|* [[srst2]]
 
|* [[SSPACE]]
 
|* [[SSPACE]]
|-
 
 
|* [[stacks]]
 
|* [[stacks]]
 
|* [[Thor]]
 
|* [[Thor]]
 +
|-
 
|* [[Tophat]]
 
|* [[Tophat]]
 
|* [[trimmomatic]]
 
|* [[trimmomatic]]
 
|* [[Trinity]]
 
|* [[Trinity]]
 
|* [[t-coffee]]
 
|* [[t-coffee]]
|-
 
 
|* [[Unicycler]]
 
|* [[Unicycler]]
 
|* [[velvet]]
 
|* [[velvet]]
 +
|-
 
|* [[ViennaRNA]]
 
|* [[ViennaRNA]]
 
|}
 
|}
Line 129: Line 136:
 
* [[Maximum Likelihood]]
 
* [[Maximum Likelihood]]
 
* [[SNP Analysis and phylogenetics]]
 
* [[SNP Analysis and phylogenetics]]
 +
* [[Normalization]]
  
 
= Pipelines =
 
= Pipelines =
(Workflow with a specific end-goal)
+
(Workflow with specific end-goals)
 
* [[Trinity_Protocol]]
 
* [[Trinity_Protocol]]
 
* [[STAR BEAST]]
 
* [[STAR BEAST]]
 
* [[callSNPs.py]]
 
* [[callSNPs.py]]
 +
* [[pairwiseCallSNPs]]
 
* [[mapping.py]]
 
* [[mapping.py]]
 
* [[Edgen RNAseq]]
 
* [[Edgen RNAseq]]
Line 142: Line 151:
 
* [[ChIP-Seq Top2 in Yeast]]
 
* [[ChIP-Seq Top2 in Yeast]]
 
* [[ChIP-Seq Top2 in Yeast 12.09.2017]]
 
* [[ChIP-Seq Top2 in Yeast 12.09.2017]]
 +
* [[ChIP-Seq Top2 in Yeast 07.11.2017]]
 +
* [[Bisulfite Sequencing]]
 +
* [[microRNA and Salmo Salar]]
  
 
=Protocols=
 
=Protocols=
Line 156: Line 168:
 
= Cluster Administration =
 
= Cluster Administration =
 
* [[Hardware Issues]]
 
* [[Hardware Issues]]
 +
* [[marvin and IPMI (remote hardware control)]]
 +
* [[restart a node]]
 
* [[Admin Tips]]
 
* [[Admin Tips]]
 
* [[RedHat]]
 
* [[RedHat]]
Line 163: Line 177:
 
* [[Blas Libraries]]
 
* [[Blas Libraries]]
 
* [[CMake]]
 
* [[CMake]]
 +
* [[conda bioconda]]
 
* [[Users and Groups]]
 
* [[Users and Groups]]
 +
* [[Installing software on marvin]]
 
* [[emailing]]
 
* [[emailing]]
 
* [[biotime machine]]
 
* [[biotime machine]]
Line 169: Line 185:
 
* [[node1 issues]]
 
* [[node1 issues]]
 
* [[6TB storage expansion]]
 
* [[6TB storage expansion]]
 +
* [[PIs storage sacrifice]]
 +
* [[SAN relocation task]]
 
* [[Home directories max-out incident 28.11.2016]]
 
* [[Home directories max-out incident 28.11.2016]]
 
* [[Frontend Restart]]
 
* [[Frontend Restart]]
Line 176: Line 194:
 
* [[Bioinformatics Wordpress Site]]
 
* [[Bioinformatics Wordpress Site]]
 
* [[Backups]]
 
* [[Backups]]
 +
* [[users disk usage]]
 +
* [[Updating BLAST databases]]
 
* [[Python DRMAA]]
 
* [[Python DRMAA]]
 +
* [[message of the day]]
 
* [[SAN disconnect incident 10.01.2017]]
 
* [[SAN disconnect incident 10.01.2017]]
 
* [[Memory repair glitch 16.02.2017]]
 
* [[Memory repair glitch 16.02.2017]]
Line 217: Line 238:
 
* [[Differential Expression Exercise]]
 
* [[Differential Expression Exercise]]
 
* [[Functional Analysis Exercise]]
 
* [[Functional Analysis Exercise]]
 +
 +
= Introduction to Unix 2017 =
 +
* [[Introduction_to_Unix_2017]]
  
  

Revision as of 09:38, 1 February 2019

Usage of Cluster

Documented Programs

The following can be seen as extra notes referring to these programs usage on the marvin cluster, with an emphais on example use-cases. Most, if not all, will have their own special websites, with more detailed manuals and further information.

* abacas * albacore * ariba * aspera * assembly-stats * augustus
* BamQC * bamtools * banjo * bcftools * bedtools * bgenie
* BLAST * Blat * blast2go: b2g4pipe * bowtie * bowtie2 * bwa
* BUSCO * CAFE * canu * cd-hit * cegma * clustal
* cramtools * conda * deeptools * detonate * diamond * ea-utils * ensembl
* ETE * FASTQC and MultiQC * Archaeopteryx and Forester * GapFiller * GenomeTools * gubbins
* JBrowse * kallisto * kentUtils * last * lastz * macs2
* Mash * mega * meryl * MUMmer * NanoSim * nseq
* OrthoFinder * PASA * perl * PGAP * picard-tools * poRe
* poretools * prokka * pyrad * python * qualimap * quast
* qiime2 * R * RAxML * Repeatmasker * Repeatmodeler * rnammer
* roary * RSeQC * samtools * Satsuma * sickle * SPAdes
* squid * sra-tools * srst2 * SSPACE * stacks * Thor
* Tophat * trimmomatic * Trinity * t-coffee * Unicycler * velvet
* ViennaRNA

Queue Manager Tips

A cluster is a shared resource with different users running different types of analyses. Nearly all clusters use a piece of software called a queue manager to fairly share out the resource. The queue manager on marvin is called Grid Engine, and it has several commands available, all beginning with q and with qsub being the most commonly used as it submits a command via a jobscript to be processed. Here are some tips:

Data Examples

Procedures

(short sequence of tasks with a certain short-term goal, often, a simple script)

Navigating genomic data websites

Explanations

Pipelines

(Workflow with specific end-goals)

Protocols

(Extensive workflows with different with several possible end goals)

Tech Reviews

Cluster Administration

Courses

I2U4BGA

hdi2u

The half-day linux course held on 20th April. Modified version of I2U4BGA.

RNAseq for DGE

Introduction to Unix 2017


Templates