Bottlenose dolphin population genomic analysis - Revision history

Rf at 16:07, 14 March 2017

2017-03-14T16:07:17Z

Rf at 15:38, 9 January 2017

2017-01-09T15:38:42Z

Rf at 15:38, 9 January 2017

2017-01-09T15:38:29Z

Rf at 15:37, 9 January 2017

2017-01-09T15:37:43Z

Rf at 15:32, 9 January 2017

2017-01-09T15:32:27Z

Rf at 15:22, 9 January 2017

2017-01-09T15:22:24Z

Rf at 15:11, 9 January 2017

2017-01-09T15:11:31Z

Rf at 14:29, 9 January 2017

2017-01-09T14:29:12Z

Rf at 13:09, 9 January 2017

2017-01-09T13:09:52Z

Rf at 13:08, 9 January 2017

2017-01-09T13:08:52Z

@@ Line 25: / Line 25: @@
 * FQN1, the forward paired output from Trimmomatic
 * FQN2, the reverse paired output from Trimmomatic
-* UBAM, the unmapped bam reflecting exclusion of mitochondiral mappings
+* MTSAM, the raw sam alignment to mito, unsorted
-MTBAM=${SSD1}/${MDN}/${SSD1}_${SRN1}_${SLN1}_to_mt_sorted.bam # the alignment to mito, sorted and now in bam format.
+* MTBAM, the alignment to mito, sorted and now in bam format.
-MTSAM=${SSD1}/${MDN}/mito_aln-pe${SSD1}_${SRN1}_${SLN1}.sam # the raw sam alignment to mito, unsorted
 TFQ1=${SSD1}/${SDN}/${SSD1}_${SRN1}_${SLN1}_forward_paired.fq.gz # trimmed fq data file, forward paired reads
 TFQ2=${SSD2}/${SDN}/${SSD2}_${SRN2}_${SLN2}_reverse_paired.fq.gz # trimmed fq data file, reverse paired reads

@@ Line 16: / Line 16: @@
 * LEADING
 * TRAILING
-* SLIDING
+* SLIDINGWINDOW
 * MINLEN

@@ Line 12: / Line 12: @@
 <ins>Explanation</ins>:
-* There are six arguments unattached to a particular option, consisting of the 2 inputs and the four outputs (because we will have paired and unpaired trimmed datasets). They are unattached because their exactposition and order is assumed to be as above.
+* There are six arguments unattached to a particular option, consisting of the 2 inputs and the four outputs (because we will have paired and unpaired trimmed datasets). They are unattached because their exact position and order is assumed to be as above.
 * ILLUMINACLIP
+* LEADING
+* TRAILING
+* SLIDING
 == Map to mitochondrial reference genome ==

@@ Line 71: / Line 71: @@
 As usual, the picard-tools function will be used for this. It can operate on multiple BAM files in one run, so we may apply it on a per-sample basis because we have several datasets for each sample in this pipeline.
-  java -Xmx32G -jar $PICARDJARFILE MarkDuplicates REMOVE_DUPLICATES=true I=<bamfile1> I=<bamfile2> I=... OPTICAL_DUPLICATE_PIXEL_DISTANCE=2500 O=<un-duplicated_bamfile> M=<marked_dup_metrics.txt>
+  java -Xmx32G -jar $PICARDJARFILE MarkDuplicates REMOVE_DUPLICATES=true I=<bamfile1> I=<bamfile2> I=... OPTICAL_DUPLICATE_PIXEL_DISTANCE=2500 O=<de-duplicated_bamfile> M=<marked_dup_metrics.txt>
 Both the '''O''' (output) and '''M''' (metrics) options (filenames with relative directory paths) are chosen by the user.
 == Realigning after duplicate identification ==
@@ Line 79: / Line 83: @@
 An initial "target creator" step is required first because the '''IndelRealigner''' has a '''-targetIntervals''' option.
-  java -Xmx16G -jar /path/to/GenomeAnalysisTK.jar -T RealignerTargetCreator -R ${REFSEQLOC} -I <unduplicated_bamfile> -o <chosen_targetcreator_intervals_filename>
+  java -Xmx16G -jar /path/to/GenomeAnalysisTK.jar -T RealignerTargetCreator -R ${REFSEQLOC} -I <de-duplicated_bamfile> -o <chosen_targetcreator_intervals_filename>
 Now the '''IndelRealigner''' tool can be invoked:
-  java -Xmx16G -Djava.io.tmpdir=/tmp -jar /path/to/GenomeAnalysisTK.jar -T IndelRealigner -R ${REFSEQLOC} -targetIntervals <chosen_targetcreator_intervals_filename> -I <unduplicated_bamfile> -o <chosen_realigned_bamfile>
+  java -Xmx16G -Djava.io.tmpdir=/tmp -jar /path/to/GenomeAnalysisTK.jar -T IndelRealigner -R ${REFSEQLOC} -targetIntervals <chosen_targetcreator_intervals_filename> -I <de-duplicated_bamfile> -o <chosen_realigned_bamfile>
 An extra bit of quality filtering is also included here:
@@ Line 93: / Line 97: @@
   intersectBed -abam <realigned_qualityfiltered_bamfile> -b <bedfile> -v | samtools view -h - | samtools view -bSh - > <no_repeats_bamfile>

@@ Line 12: / Line 12: @@
 <ins>Explanation</ins>:
-* There are six arguments unattached to a particular option, consisting of the 2 inputs and the four outputs (because we will have paired and unpaired trimmed datasets)
+* There are six arguments unattached to a particular option, consisting of the 2 inputs and the four outputs (because we will have paired and unpaired trimmed datasets). They are unattached because their exactposition and order is assumed to be as above.
@@ Line 35: / Line 36: @@
 First we align to the mitochondrial reference:
-  bwa mem -t $NSLOTS $MITOREFLOC $TFQ1 $TFQ2 > $MTSAM
+  bwa mem -t $NSLOTS $MITOREFLOC <quality_trimmed_forward_paired>  <quality_trimmed_reverse_paired> > <alignedtomitochondrial_samfile>
 The resulting sam is sorted and converted to bam format:
-  samtools view -bSh $MTSAM | samtools sort - ${MTBAM%.*} -@ $NSLOTS
+  samtools view -bSh <alignedtomitochondrial_samfile> | samtools sort - <alignedtomitochondrial_bamfile_withoutextension> -@ $NSLOTS
-Actually we're only interested in the reads that did not map to the mitochondrial, and we can extract them into the their own bam with:
+Note that the ''''.bam''' extension is not required in the output of samtools sort when its input and output are bam.
-  samtools view -f4 -bh $MTBAM > $UBAM -@ $NSLOTS
+== Re-generate readsets which will no longer map to mitochondrial genome ==
 <ins>Explanation</ins>:

@@ Line 58: / Line 58: @@
   bwa mem -t $NSLOTS $REFLOC $FQN1 $FQN2 > $REASAM
-The raw sam can be sorted and converted to bam to produce the alignment with the reads mapping to teh mitochondiral reference excluded as follows:
+The raw sam can be sorted and converted to bam to produce the alignment with the reads mapping to the mitochondrial reference excluded as follows:
   samtools view -bSh ${REASAM} | samtools sort - ${REABAM%.*} -@ $NSLOTS
@@ Line 83: / Line 83: @@
   samtools view -bh -F4 -q 30 <chosen_realigned_bamfile> <realigned_qualityfiltered_bamfile> -@ $NSLOTS
-== Intersecting for features ==
+== Repeatmasking using a bed file ==
   intersectBed -abam <realigned_qualityfiltered_bamfile> -b <bedfile> -v | samtools view -h - | samtools view -bSh - > <no_repeats_bamfile>

@@ Line 7: / Line 7: @@
 == Quality trimming ==
-Using trimmomatic
+Using trimmomatic, which allows various quality trimming options
 MITOREFLOC=/storage/home/users/ml228/Genomics/NEA_BGI_data_ALL/NEA_BGI_clean_raw_data/Refs/mitogenome/gi_557468684_gb_KF570351.1_.fasta

← Older revision		Revision as of 13:09, 9 January 2017
Line 71:		Line 71:

	An extra bit of quality filtering is also included here:		An extra bit of quality filtering is also included here:
−	samtools view -bh -F4 -q 30 ~~${SSD}/${TSD}/Ind_${SSDND}_U_al.bam~~ > ~~${SSD}/${TSD}/${SSDND}_q30.bam~~ -@ $NSLOTS	+	samtools view -bh -F4 -q 30 <chosen_realigned_bamfile> <realigned_qualityfiltered_bamfile> -@ $NSLOTS
−	~~# sequentially that could take an hour at least.~~